UW researchers designed a headphone system that translates several people speaking at once, following them as they move and preserving the direction and qualities of their voices. The team built the...
  • yesman@lemmy.worldEnglish
    4·
    1 month ago

    I hate AI as much as the next lemming, but nobody is going to tell me a babble fish for real isn’t cool AF.

  • stoy@lemmy.zipEnglish
    1·
    1 month ago

    Ok, so this concept is cool, but has a few problems…

    1. Privacy, this is far too complex to run on the headphones themselves, so the system will need to connect to a server to do the heavy lifting, what happens to the data once it used? For legal purposes I suspect it will need to be saved, meaning that any thing recorded could be analyzed or monitored.
    2. Trust, AI models have rules in place to make them act in specific ways, the owner of the AI system used could tweak it to change what spoken or how it is said, this could push political agendas in everyday conversations.
    3. Reduced lingual skills, an AI like this would reduce the incentive to learn another language, reducing people’s international direct communications, increasing dependancy on the AI service, further reducing our lingual skills.

    This is scary…

    • iturnedintoanewt@lemm.eeEnglish
      2·
      1 month ago

      Check whisper apk on fdroid. The thing runs local. It does just this. The model gets audio in an undetermined language, figures out which one automatically, transcribes it, translates it to English (only English atm) and then it speaks it out. It’s not using any acceleration and its a very early build. My Pixel 9 is getting about 3 seconds delay from input to output. It’s all running local.

      It’s doable.

    • lakemalcom10@lemm.eeEnglish
      2·
      1 month ago

      For 1 they actually addressed that: The system then translates the speech and maintains the expressive qualities and volume of each speaker’s voice while running on a device, such mobile devices with an Apple M2 chip like laptops and Apple Vision Pro. (The team avoided using cloud computing because of the privacy concerns with voice cloning.) Finally, when speakers move their heads, the system continues to track the direction and qualities of their voices as they change.

  • gedaliyah@lemmy.worldEnglish
    1·
    1 month ago

    Anyone who likes this idea might also be interested in checking out RTranslator, an open source, on-device app, which has some similar functionality. You can connect two Bluetooth devices using this app to communicate between two people in different languages.

    It can’t translate multiple speakera simultaneously or clone voices, but it’s very useful for traveling or communicating with friends and family in multiple languages. Especially since it does not need any connection, it comes in handy on the road when you might not have a reliable connection.