Privacy-Aware Compression for Federated Learning Through Numerical Mechanism Design. (arXiv:2211.03942v2 [cs.LG] UPDATED)

In private federated learning (FL), a server aggregates differentially
private updates from a large number of clients in order to train a machine
learning model. The main challenge in this setting is balancing privacy with
both classification accuracy of the learnt model as well as the number of bits
communicated between the clients and server. Prior work has achieved a good
trade-off by designing a privacy-aware compression mechanism, called the
minimum variance unbiased (MVU) mechanism, that numerically solves an
optimization problem to determine the parameters of the mechanism. This paper
builds upon it by introducing a new interpolation procedure in the numerical
design process that allows for a far more efficient privacy analysis. The
result is the new Interpolated MVU mechanism that is more scalable, has a
better privacy-utility trade-off, and provides SOTA results on
communication-efficient private FL on a variety of datasets.