• 0 Posts
  • 21 Comments
Joined 1 year ago
cake
Cake day: July 8th, 2023

help-circle

  • Oh I suppose it’s still not as bad as other places. It was free and fully financed by taxes because it was seen as a societal investment, now it’s ~300€ per semester at my university and still mostly financed through taxes. But the neoliberals sure didn’t impose that without a fight! And they would probably have set it higher if they could.

    I mean I know I’m complaining from a position of privilege here (sorry US debt slave bros), but still, fuck that shit. Cutting the most financially vulnerable people in society out of an education is what that amounts to. In other words, it’s just another front in class warfare.





  • a neural network with a series of layers (W in this case would be a single layer)

    I understood this differently. W is a whole model, not a single layer of a model. W is a layer of the Transformer architecture, not of a model. So it is a single feed forward or attention model, which is a layer in the Transformer. As the paper says, a LoRA:

    injects trainable rank decomposition matrices into each layer of the Transformer architecture

    It basically learns shifting the output of each Transformer layer. But the original Transformer stays intact, which is the whole point, as it lets you quickly train a LoRA when you need this extra bias, and you can switch to another for a different task easily, without re-training your Transformer. So if the source of the bias you want to get rid off is already in these original models in the Transformer, you are just fighting fire with fire.

    Which is a good approach for specific situations, but not for general ones. In the context of OP you would need one LoRA for fighting it sexualising Asian women, then you would need another one for the next bias you find, and before you know it you have hundreds and your output quality has degraded irrecoverably.


  • Yeah but that’s my point, right?

    That

    1. you do not “replace data until your desired objective”.
    2. the original model stays intact (the W in the picture you embedded).

    Meaning that when you change or remove the LoRA (A and B), the same types of biases will just resurface from the original model (W). Hence “less biased” W being the preferable solution, where possible.

    Don’t get me wrong, LoRAs seem quite interesting, they just don’t seem like a good general approach to fighting model bias.


  • First, there is no thing as a “de-biased” training set, only sets with whatever target series of biases you define for them to reflect.

    Yes, I obviously meant “de-biased” by definition of whoever makes the set. Didn’t think it worth mentioning, as it seems self evident. But again, in concrete terms regarding the OP this just means not having your dataset skewed towards sexualised depictions of certain groups.

    1. either you replace data until your desired objective, which will reduce the model’s quality for any of the alternatives

    […]
    For reference, LoRAs are a sledgehammer approach to apply the first way.

    The paper introducing LoRA seems to disagree (emphasis mine):

    We propose Low-Rank Adaptation, or LoRA, which freezes the pre-trained model weights and injects trainable rank decomposition matrices into each layer of the Transformer architecture, greatly reducing the number of trainable parameters for downstream tasks.

    There is no data replaced, the model is not changed at all. In fact if I’m not misunderstanding it adds an additional neural network on top of the pre-trained one, i.e. it’s adding data instead of replacing any. Fighting bias with bias if you will.

    And I think this is relevant to a discussion of all models, as reproduction of training set biases is something common to all neural networks.


  • “Inclusive models” would need to be larger.

    [citation needed]

    To my understanding the problem is that the models reproduce biases in the training material, not model size. Alignment is currently a manual process after the initial unsupervised learning phase, often done by click-workers (Reinforcement Learning from Human Feedback, RLHF), and aimed at coaxing the model towards more “politically correct” outputs; But ultimately at that time the damage is already done since the bias is encoded in the model weights and will resurface in the outputs just randomly or if you “jailbreak” enough.

    In the context of the OP, if your training material has a high volume of sexualised depictions of Asian women the model will reproduce that in its outputs. Which is also the argument the article makes. So what you need for more inclusive models is essentially a de-biased training set designed with that specific purpose in mind.

    I’m glad to be corrected here, especially if you have any sources to look at.









  • All true, and maybe I have been understating my negative estimation of the whole ordeal a bit, but we are like five levels of escalation deep at this point, and all I’m saying is it’s getting harder and harder to gain a nuanced understanding of the situation. Which is important.

    Anyway, given the totality of what I have seen so far LTT has either always been or just devolved into an entirely toxic work environment. Their reaction so far doesn’t inspire any confidence in the slightest. On the contrary, it reinforces all of the accusations.


  • This isn’t “something”. This is a “we’re sorry (not really)” video. If you watch it in the context of “I have no favorites in this game” and look it it pretty objectively, it feels like just a bait to try to stop the bleeding.

    Yeah that’s what I meant with “Southpark-y ‘I’m sorry’ vibes”. For reference: https://www.youtube.com/watch?v=15HTd4Um1m4

    P.S.: And that’s… “something”.

    A week? To refine all the processes in that size of a company?

    No, a week without videos to get started with reviewing processes. I agree with you in general though, if it stops there it’s nothing more than PR. Remains to be seen what will come of it, but the allegations by that former employee are certainly a dampener on an optimistic view of the situation.



  • How is a language based approach that completely abstracts away actual knowledge, and just tries to sound “good enough” any kind of useful in a medical workflow?

    A LLM cross-referencing a list of symptoms against papers and books could be helpful for example. There is so much medical literature available these days and in so many languages that no one person can hope to gain a somewhat clear overview, much less keep up with all the new stuff coming out.

    Of course this should only be in assistance to a trained medical professional, as all neural networks are prone to hallucinations. You should also double-check results of NNs that interpret medical images, they may straight-up hallucinate or just pick up on correlation instead of causation (say all the cancer images in your training set having a watermark from the same lab or equipment manufacturer).