Its one thing to have an academic approach to synthesis and sound design - its another thing entirely to have the funds to get ones own lab up and running and have some crazy AI thing built for you:
"
For example, a user might begin with a collection of thousands or even millions of books and PDFs, turn on some filters for specific features like “mention of geopolitical conflict” and “escalating rhetoric”, then quickly zoom in to the highlighted parts to find relevant passages and paragraphs. Compared to a conventional search that flattens all the detail into a single sorted list of a few dozen items, a heatmap lets the user see detail without getting lost in it.
"
This already exists in a few Ask your PDF tools, and I haven't found it useful.
The problem in text is not discoverability. Order just matters too much in text. Just like how plucking exact match search results is about as useful as semantic search for 99% of use cases, the context matters. You either have something you want or you don't. The issue with semantic matching and trying to display or use that information is that you end up with a lot of blur and would be better of feeding such information to a GPT model to summarize, or simply using a GPT model initially, to construct something more useful for you to work with. Or in other words - semantic comparisons and grouping of content creates too much bloat compared to the reasoning currently possible by GPT models, and there is no solution to this here.
A concept library for sharing and collaborating on units of meaning and style
I've been pining for this for so long.
For example, I'd love to make a todo app where the user doesn't need to worry about the exact wording when they add or search for something; where they can choose the amount of verbosity in the UI... but not be stuck with a todo database that can't easily be imported to a different app.
It would be fantastic for a new kind of webbrowser, too: smaller file sizes, and perfect support for all human languages.
I can see the appeal in "highlight by theme" going one way.
I can't see the appeal in synthesizing from theme; music it doesn't matter if there's more drums or more trumpet, if it's brighter or more mournful, the details don't really matter - the mood and feeling matters. But text isn't like that for me, words have meanings, they come one after another, I don't think I want synthesized writing which has more "concept of being held" and less "discussions about parenthood"; what would it be - poetry? beat poetry? SEO marketing fluff which tries to convey a feeling without being nailed down to any specific claims or accuracy? That kind of writing is something to cut through to get to what's behind it and tiring to do it; Who would want to spend time reading it for pleasure like people listen to music for pleasure, and in what situations?
It starts with 'synthesizer for thought' but moves to synthesizer for words and writing. We already understand the concepts they mention, American Robin, being held, there being three of something. New synthesized thoughts as words, how do they become more meaningful than just mishmashing words normally - "a 13 dimensional concrete", "a society based on <untranslatable>", "French, but it's whalesong"?
From text. That connection has been investigated for a long time.
Although I must admit, it isn't entirely clear what the author is talking about. It's a bit like thoughts that appear when one is somewhat intoxicated. But extracting a mathematical model from linguistic utterances is very much what linguistics is about (and hasn't been able to achieve). Perhaps rotating the representational space of an LLM suits his/her fancy.
I wonder what Aphex Twin has to say about this?
Its one thing to have an academic approach to synthesis and sound design - its another thing entirely to have the funds to get ones own lab up and running and have some crazy AI thing built for you:
https://www.factmag.com/2017/07/14/watch-aphex-twin-midimuta...
Would be curious how these two projects collide/coallesce ..
" For example, a user might begin with a collection of thousands or even millions of books and PDFs, turn on some filters for specific features like “mention of geopolitical conflict” and “escalating rhetoric”, then quickly zoom in to the highlighted parts to find relevant passages and paragraphs. Compared to a conventional search that flattens all the detail into a single sorted list of a few dozen items, a heatmap lets the user see detail without getting lost in it. "
This already exists in a few Ask your PDF tools, and I haven't found it useful.
The problem in text is not discoverability. Order just matters too much in text. Just like how plucking exact match search results is about as useful as semantic search for 99% of use cases, the context matters. You either have something you want or you don't. The issue with semantic matching and trying to display or use that information is that you end up with a lot of blur and would be better of feeding such information to a GPT model to summarize, or simply using a GPT model initially, to construct something more useful for you to work with. Or in other words - semantic comparisons and grouping of content creates too much bloat compared to the reasoning currently possible by GPT models, and there is no solution to this here.
For example, I'd love to make a todo app where the user doesn't need to worry about the exact wording when they add or search for something; where they can choose the amount of verbosity in the UI... but not be stuck with a todo database that can't easily be imported to a different app.
It would be fantastic for a new kind of webbrowser, too: smaller file sizes, and perfect support for all human languages.
I believe this article is a prime example of an intuition pump. I'm literally a different person after reading it.
I can see the appeal in "highlight by theme" going one way.
I can't see the appeal in synthesizing from theme; music it doesn't matter if there's more drums or more trumpet, if it's brighter or more mournful, the details don't really matter - the mood and feeling matters. But text isn't like that for me, words have meanings, they come one after another, I don't think I want synthesized writing which has more "concept of being held" and less "discussions about parenthood"; what would it be - poetry? beat poetry? SEO marketing fluff which tries to convey a feeling without being nailed down to any specific claims or accuracy? That kind of writing is something to cut through to get to what's behind it and tiring to do it; Who would want to spend time reading it for pleasure like people listen to music for pleasure, and in what situations?
It starts with 'synthesizer for thought' but moves to synthesizer for words and writing. We already understand the concepts they mention, American Robin, being held, there being three of something. New synthesized thoughts as words, how do they become more meaningful than just mishmashing words normally - "a 13 dimensional concrete", "a society based on <untranslatable>", "French, but it's whalesong"?
I love Linus' blogs
While this is kind of interesting, it just looks like it's a long way round to recreate Corpus Linguistics: https://en.wikipedia.org/wiki/Corpus_linguistics
It seems to ignore a couple of millenia of linguistics.
Linguistics isn't thought. I believe the author is specifically talking about a latent representation of conversation.
From text. That connection has been investigated for a long time.
Although I must admit, it isn't entirely clear what the author is talking about. It's a bit like thoughts that appear when one is somewhat intoxicated. But extracting a mathematical model from linguistic utterances is very much what linguistics is about (and hasn't been able to achieve). Perhaps rotating the representational space of an LLM suits his/her fancy.