Federated represents a break from the traditional way of gathering data and training machine learning models.
With federated learning, machine learning development benefits from cheaper training that respects data privacy. This article guides what federated learning is, how it works, applications, and frameworks.
What is Federated Learning?
Federated Learning is a shift in how machine learning models are trained. In most machine learning models, data is gathered into a central repository from several clients. From this central repository, machine learning models are trained and then used to make predictions. Federated Learning does the opposite. Instead of sending data to a central repository, clients train models on their data. This helps them keep their private data private.
Also read: Top Machine Learning Models Explained
How Federated Learning Works?
Learning in Federated Learning comprises a series of atomic steps that produce a model. These steps are called learning rounds. A typical learning setup iterates through these rounds, improving the model at each step. Each learning round involves the following steps.
A Typical Learning Round
First, the server chooses the model to train and hyperparameters such as the number of rounds, client nodes to use, and the fraction of nodes used at each node. At this point, the model is also initialized with the initial parameters to form the base model.
Next, clients get copies of a base model to train. These clients could be mobile devices, personal computers, or servers. They train the model on their local data, thus avoiding sharing sensitive data with the servers.
Once clients have trained the model on their local data, they send it back to the server as an update. When received by the server, the update is averaged alongside updates from other clients to create a new base model. Since clients can be unreliable, at this point, some clients may not send in their updates. At this point, the server will handle all errors.
Before the base model can be redeployed, it has to be tested. However, the server does not store data. Therefore, to test the model, it is sent back to the clients, where it is tested against their local data. If it is better than the previous base model, it is adopted and used instead.
Here’s a helpful guide on how federated learning works from the Federated Learning team at Google AI.
Centralized vs. Federated vs. Heterogenous
In this setup, there is a central server responsible for controlling the learning. This type of setup is known as Centralised Federated Learning.
The opposite of Centralised Learning would be Decentralised Federated Learning, in which clients coordinate themselves peer-to-peer.
The other setup is called Heterogenous Learning. In this setup, clients do not necessarily have the same global model architecture.
Advantages of Federated Learning
- The greatest advantage of using federated learning is that it helps keep private data private. Clients share the results of training, not the data used in training. Protocols can also be put in place to aggregate results so they cannot be linked to a particular client.
- It also reduces the network bandwidth as no data is shared between the client and server. Instead, models are exchanged between the client and server.
- It also reduces the cost of training models, as there is no need to buy expensive training hardware. Instead, developers make use of the client’s hardware to train models. Because of the little data involved, it does not strain the client’s device.
Disadvantages of Federated Learning
- This model depends on the participation of many different nodes. Some of which are not controlled by the developer. Therefore, their availability is not guaranteed. This makes the training hardware unreliable.
- The clients on which models are trained are not exactly powerful GPUs. Instead, they are normal devices such as phones. These devices, even in aggregate, might not be powerful enough compared to GPU clusters.
- Federated learning also assumes that all the client nodes are trustworthy and work for the common good. However, some might not, and they might issue bad updates to cause model drift.
Applications of Federated Learning
Federated Learning enables learning while preserving privacy. This is useful in many situations, such as:
- Next-word predictions on smartphone keyboards.
- IoT devices that can train models locally on the specific requirements of the situation they are in.
- Pharmaceuticals and healthcare industries.
- Defense industries would also benefit from training models without sharing sensitive data.
Frameworks for Federated Learning
There are many frameworks for implementing Federated Learning patterns. Some of the best ones include NVFlare, FATE, Flower and PySft. Read this guide for a detailed comparison of the different frameworks you can use.
Conclusion
This article was an introduction to Federated Learning, how it works, as well as the advantages and disadvantages of implementing it. In addition, I also covered the popular applications and frameworks used to implement Federated Learning in production.
Next, read an article on the best MLOps platforms to train your Machine Learning models.
