Fully-distributed optimization with Network Exact Consensus-GIANT

We consider a fully-distributed optimization problem involving multiple collaborative agents, where the global objective is to minimize a sum of local cost functions. Agents are part of a communication network and can only exchange information with their neighbors. We introduce a novel optimization algorithm called NEC-GIANT, which improves over both GIANT, a popular federated learning algorithm, and Network-GIANT, our previously proposed fully-distributed counterpart of GIANT. NEC-GIANT extends GIANT to the fully-distributed scenario, removing the need for a central server to orchestrate the agents. Unlike the existing Network-GIANT, which suffers from the inefficiency of standard asymptotic consensus, the novel NEC-GIANT is based on finite-time distributed consensus and retains all the convergence properties of the original GIANT. Numerical simulations prove the efficiency and superiority of the proposed algorithm in terms of both iterations and machine run-time.