I created a utility for memoization of parameter sweeps called chunk-memo. It collects multiple runs to be written to single files for increased disk read/write efficiency (chunking) including parallelization. github.com/msf235/chunk-memo.
2
59
My paper with @CPehlevan is out now in PNAS! doi.org/10.1073/pnas.2309876… Sequences are a core part of an animal's behavioral repertoire, and Hebbian learning allows neural circuits to store memories of sequences for later recall.
It’s become trendy to use trained neural networks as models for the brain. New theory is helping us appreciate the different approaches neural networks will use to solve tasks depending on initialization, optimization, and other details. Our review here: authors.elsevier.com/c/1hplN…
Here's a paper @CPehlevan and I wrote about neural networks that generate sequences: biorxiv.org/content/10.1101/…. Hebbian learning can store sequences as memories in neural networks for later recall. However, the tempo of the recalled sequence can differ from that of the tutor signal
A long time ago I wrote a simple utility for managing the output of my simulations. I've finally gone to the trouble of cleaning it up enough to share: github.com/msf235/model_outp…. This utility keeps a log of simulation parameters in a .csv file for simple but flexible memoization.
3
New preprint arxiv.org/abs/2110.07472. We look at the expressivity of neural representations that are equivariant to their inputs -- that is, as input objects transform according to a group action, the representations transform according to an action of the same group.
This reveals that the expressivity of standard convolutional network layers (in the case where inputs are shown at all possible cyclic shifts) is determined by the number of filters/output channels. Here is the expressivity/capacity of VGG11 as the number of filters varies.
1
1
It's sometimes surprising (e.g. Sorscher et al. https://nitter.cf/t.co/pNDaKVbjv7) how even really simple circuits trained with simple loss functions can reveal interesting phenomena. The final version of some of my work at UW is now up (https://nitter.cf/t.co/Ze6v9cijif).
Here we look at a one-layer autoencoder model and ask what can be said about the structure of the connectivity matrices. We find that nonlinear dim reduction is a natural approach for extracting this structure, and that the appearance of structure depends on regularization.
2
1

