Matthew Farrell (@MattSFarrell) profile picture

Special Postdoctoral Researcher at the Laboratory for Neural Computation and Adaptation, RIKEN Center for Brain Science

Joined February 2019
Photo by Matthew Farrell (@MattSFarrell)Photo by Matthew Farrell (@MattSFarrell)Photo by Matthew Farrell (@MattSFarrell)
I created a utility for memoization of parameter sweeps called chunk-memo. It collects multiple runs to be written to single files for increased disk read/write efficiency (chunking) including parallelization. github.com/msf235/chunk-memo.
2
59
My paper with @CPehlevan is out now in PNAS! doi.org/10.1073/pnas.2309876… Sequences are a core part of an animal's behavioral repertoire, and Hebbian learning allows neural circuits to store memories of sequences for later recall.
2
10
2
31
5,216
Interestingly, the tempo of a recalled sequence can differ from that of the "tutor signal" used to teach the network. We show how this works, relating the shape of the STDP kernel and the tutor signal tempo to the dynamics of the recalled sequence, including its own tempo.
351
It’s become trendy to use trained neural networks as models for the brain. New theory is helping us appreciate the different approaches neural networks will use to solve tasks depending on initialization, optimization, and other details. Our review here: authors.elsevier.com/c/1hplN…
1
10
39
5,960
In this work we show how the tempo of the recalled sequences depends on the Hebbian learning kernel (such as the shape of an STDP curve) and the duration for which each step in the sequence is shown to the network by the tutor signal.
6
363
A long time ago I wrote a simple utility for managing the output of my simulations. I've finally gone to the trouble of cleaning it up enough to share: github.com/msf235/model_outp…. This utility keeps a log of simulation parameters in a .csv file for simple but flexible memoization.
3
New preprint arxiv.org/abs/2110.07472. We look at the expressivity of neural representations that are equivariant to their inputs -- that is, as input objects transform according to a group action, the representations transform according to an action of the same group.
4
7
2
32
This reveals that the expressivity of standard convolutional network layers (in the case where inputs are shown at all possible cyclic shifts) is determined by the number of filters/output channels. Here is the expressivity/capacity of VGG11 as the number of filters varies.
1
1
However, convolutional architectures built around other representations of the cyclic group can have higher expressivity. See the preprint for more info.
1
It's sometimes surprising (e.g. Sorscher et al. https://nitter.cf/t.co/pNDaKVbjv7) how even really simple circuits trained with simple loss functions can reveal interesting phenomena. The final version of some of my work at UW is now up (https://nitter.cf/t.co/Ze6v9cijif).
1
1
9
Here we look at a one-layer autoencoder model and ask what can be said about the structure of the connectivity matrices. We find that nonlinear dim reduction is a natural approach for extracting this structure, and that the appearance of structure depends on regularization.
2
1
The link in my first tweet gives full access to the paper until June 22.