|
|---|
Distributed vector representations are a key bridging point between connectionist and symbolic representations of cognition. It is unclear how uncertainty should be modelled in systems using such representations. We discuss how bundles of symbols in Vector Symbolic Architectures (VSAs) can be understood as defining an object that has a relationship to a probability distribution, and how statements in VSAs can be understood as being analogous to probabilistic statements. We sketch novel designs for networks that compute entropy and mutual information of VSA-represented distributions. While we restrict ourselves to operators proposed for Holographic Reduced Representations, and representing real-valued data, we suggest that the methods presented in this paper should translate to any VSA where the dot product between fractionally bound symbols induces a valid kernel.
Michael Furlong is a Postdoctoral Research Fellow in the Computational Neuroscience Research Group at the University of Waterloo. Prior to this, Michael was a contractor with NASA Ames Research Center's Intelligent Robotics Group. Michael received his Ph.D. in robotics from CMU in 2018, a M.Sc. in Neuroscience from Oxford in 2011, a M.Sc. in Robotics from CMU in 2010, and a B.Eng. from Memorial University in 2005. Michael's research interests include Vector Symbolic Architectures and their application to efficient prediction and exploration schemes for robotic systems.