Idea Book 📔🖊️

Where do new ideas come from? Late-night conversations with friends? Moments of inspiration during talks and seminars? I’m not sure, but I do know that the process of generating ideas is like fishing. You have to be patient and be ready to catch that light bulb when it appears, or else it will just slip away to the nowhere it has come from.

If any of the ideas below resonate with you, feel free to reach out (or even to inform me that someone has already worked or is currently working on the idea) . I’d love to explore new possibilities and make them happen together. Additionally, if you are inspired to pursue or build upon any of them, I only ask that you acknowledge the inspiration.

Disclaimer: The ideas that I’m sharing might come across as futuristic, vague, or unconventional. However, I believe this is exactly what fuels innovation, a blend of bold vision and a touch of imagination.

Dynamic Neural Memory for In-Context Learning: SSMs or Transformers?

Can we combine features of both SSMs and Transformers to build a graph-like hierarchical memory mechanism with sublinear growth? Each node represents a concept, and edges connect concepts similar to an associative memory. The graph is traversed with attention mechanism to retrieve or add information.