Machine learning systems for distributed deep learning, including collaborative heterogeneous processors.