Motivation: Neural networks (NNs) are continuously being integrated into increasingly more critical aspects of society, however our understanding of the underlying organizational patterns within NNs is limited. Biological NNs are highly organized in physical space with proximal regions activating together at higher rates than non-proximal regions. In this work, we develop a set of tools to elucidate the design and organization of weights in NNs. While software tools for the creation and development of NNs are robust, there exists a shortage of tools aimed at analyzing and visualizing the distribution of weight values across large numbers of these NNs. Understanding the distributions of NN weights can provide researchers with practical insights like how to most efficiently initialize weights in order to reduce network training time. It may also help draw a more explicit link between organizational patterns in artificial and biological NNs.

Methods: Here, we present a suite of tools to generate, visualize, and compare the statistical distribution of weights across large numbers of small NNs. Key features include a pipeline to generate networks given some architecture and functions to visualize and analyze the statistical distribution of weights in networks generated by the pipeline. Weight values are partitioned into discrete bins and aggregated across networks. We then fit probability distributions to these aggregated data and compare them via KL-divergence. This allows for the identification of unique and conserved distributions of weight values in the set of networks. To date, we have trained and analyzed the distributions of weight values across >50k networks utilizing this suite of tools. Going forward, we aim to develop Bayesian tools to analyze conditional probabilities between distributions. 

Availability: The tools are available free and open-source at: github.com/sharifware/Large-Scale-Design-and-Analysis-of-Neural-Networks/