• 0 Posts
  • 78 Comments
Joined 5 months ago
cake
Cake day: June 30th, 2025

help-circle






  • NotANumber@lemmy.dbzer0.comtoScience Memes@mander.xyzSquiggly Boie
    link
    fedilink
    English
    arrow-up
    4
    ·
    edit-2
    1 month ago

    Yes genetic algorithms are something different. Though they are used sometimes in training or architecting NNs, but not at the scale of modern LLMs.

    Fyi you can have all or nothing outputs from a perceptron or other network. It all depends on the activation function. Most LLMs don’t use that kind of activation function, but it is possible. Have you heard of bitnet? They use only one of three states for the neuron output in an LLM. It’s interesting stuff.





  • NotANumber@lemmy.dbzer0.comtoScience Memes@mander.xyzSquiggly Boie
    link
    fedilink
    English
    arrow-up
    16
    ·
    edit-2
    1 month ago

    To be more specific this is an MLP (Multi-Layer Perceptron). Neural Network is a catch all term that includes other things such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Diffusion models and of course Transformers.

    What you are arguing online is some variant of a Generative Pre-trained Transformer, which do have MLP or MoE layers but that’s only one part of what they are. They also have multi-headed attention mechanisms and embedding + unembedding vectors.

    I know all this and wouldn’t call myself a machine learning expert. I just use the things. Though I did once train a simple MLP like the one in the picture. I think it’s quite bad calling yourself a machine learning expert and not knowing all of this stuff and more.




  • well that settles it then! you’re apparently such an authority.

    I am someone who is paid to research uses and abuses of AI and LLMs in a specific field. So compared to randos on the internet like you, yeah I could be considered an authority. Chances are though you don’t actually care about any of this. You just want an excuse to hate on something you don’t like and don’t understand and blame it for already well established problems. How about instead you actually take some responsibility for the state of your fellow human beings and do something helpful instead of being a Luddite.




  • I don’t trust OpenAI and try to avoid using them. That being said they have always been one of the more careful ones regarding safety and alignment.

    I also don’t need you or openai to tell me that hallucinations are inevitable. Here have a read of this:

    Title: Hallucination is Inevitable: An Innate Limitation of Large Language Models, Author: Xu et al., Date: 2025-02-13, url: http://arxiv.org/abs/2401.11817

    Regarding resource usage: this is why open weights models like those made by the Chinese labs or mistral in Europe are better. Much more efficient and frankly more innovative than whatever OpenAI is doing.

    Ultimately though you can’t just blame LLMs for people committing suicide. It’s a lazy excuse to avoid addressing real problems like how treats neurodivergent people. The same problems that lead to radicalization including incels and neo nazis. These have all been happening before LLM chatbots took off.


  • I am sure the terminal IDEs are great. I did used to play around with vim myself, and still use it for editing config files. I have had some success with Jet Brains as well. It’s a solid product.

    I don’t really have the energy it takes to configure and learn all the stuff that’s needed for a terminal only setup these days. I guess I am just not as discerning as you are. I might try a ready made solution like LazyVim.