Why my company is stuck rn

[Jason Voorhees]

I'll keep it as simply as possible so anyone can understand. You see all computers store things Os and 1s. Everything is long sequences of Os and 1s but the problem is all systems since the dawn of computers have rounding errors. 0.1+0.2 =/= 0.3 because decimal numbers cannot reliably be represented in Os and 1s.It would be something like 0.30000000000000004 or something equally cursed.

It's a fundamental law of how binary floating point (IEEE 754) works. For 50+ years we have just ignored this because who tf cares but now with AI in the picture. You can't simply ignore it. Millions of matrix multiplications per second,millisecond inferences and perfect consistency across training runs. It means even than tiny errors get magnified into something catastrophic across 175 billion parameters.

This isn't a huge problem generally neural networks don't need mathematical perfection infact gradient descent actually loves a bit of noise and garbage for generalization but the problem is here we are dealing with algorithms. There's ofc workarounds like quantization, tensor cores in GPUs specifically designed to handle this with FP32 but there's none that specifically catering to our needs because we require a deterministic bit exact reproducibility.

 

[Jason Voorhees]

@Glorious King @topology @SharpOrange @LXR @User28823

 

[NinjaRG9]

Always some bullshit with this old nigga

 

[Jason Voorhees]

@mcmentalonthemic @jeoyw9192 @Jatt @childishkillah

 

[tansel]

 

[unknownincel]

sounds complicated

dropped comp science

 

[Jason Voorhees]

@imontheloose @dhusc @unstable @Jager

 

[ER1887]

I'll keep it as simply as possible so anyone can understand. You see all computers store things Os and 1s. Everything is long sequences of Os and 1s but the problem is all systems since the dawn of computers have rounding errors. 0.1+0.2 =/= 0.3 because decimal numbers cannot reliably be represented in Os and 1s.It would be something like 0.30000000000000004 or something equally cursed.

It's a fundamental law of how binary floating point (IEEE 754) works. For 50+ years we have just ignored this because who tf cares but now with AI in the picture. You can't simply ignore it. Millions of matrix multiplications per second,millisecond inferences and perfect consistency across training runs. It means even than tiny errors get magnified into something catastrophic across 175 billion parameters.

This isn't a huge problem generally neural networks don't need mathematical perfection infact gradient descent actually loves a bit of noise and garbage for generalization but the problem is here we are dealing with algorithms. There's ofc workarounds like quantization, tensor cores in GPUs specifically designed to handle this with FP32 but there's none that specifically catering to our needs because we require a deterministic bit exact reproducibility.

I hate working with computers

 

[unstable]

what do you think about non transistor processors for ai?
i think they are all hype.

 

[mcmentalonthemic]

Will read later im playing video games

 

[Jason Voorhees]

Will read later im playing video games

Which one?

 

[Jason Voorhees]

what do you think about non transistor processors for ai?
i think they are all hype.

That's the hardware side of things that I'm not sure about tbh

 

[Jason Voorhees]

@Incelforeever

 

[unstable]

which languages would you advice someone to learn for best job opportunities.

 

[SharpOrange]

what do you think about non transistor processors for ai?
i think they are all hype.

What course is this? Electronics?

 

[SharpOrange]

which languages would you advice someone to learn for best job opportunities.

Hebrew

 

[Jason Voorhees]

which languages would you advice someone to learn for best job opportunities.

Learn python and FASTAPI

 

[mcmentalonthemic]

Which one?

ready or not

 

[Jason Voorhees]

What course is this? Electronics?

ECE but no course will teach that specific thing. You'll have to learn that yourself

 

[elliottttt]

I'll keep it as simply as possible so anyone can understand. You see all computers store things Os and 1s. Everything is long sequences of Os and 1s but the problem is all systems since the dawn of computers have rounding errors. 0.1+0.2 =/= 0.3 because decimal numbers cannot reliably be represented in Os and 1s.It would be something like 0.30000000000000004 or something equally cursed.

It's a fundamental law of how binary floating point (IEEE 754) works. For 50+ years we have just ignored this because who tf cares but now with AI in the picture. You can't simply ignore it. Millions of matrix multiplications per second,millisecond inferences and perfect consistency across training runs. It means even than tiny errors get magnified into something catastrophic across 175 billion parameters.

This isn't a huge problem generally neural networks don't need mathematical perfection infact gradient descent actually loves a bit of noise and garbage for generalization but the problem is here we are dealing with algorithms. There's ofc workarounds like quantization, tensor cores in GPUs specifically designed to handle this with FP32 but there's none that specifically catering to our needs because we require a deterministic bit exact reproducibility.

dnr

 

[unstable]

What course is this? Electronics?

nah youtube autism

 

[hyperbeast]

Hire me bhai

 

[birthdefect]

I'll keep it as simply as possible so anyone can understand. You see all computers store things Os and 1s. Everything is long sequences of Os and 1s but the problem is all systems since the dawn of computers have rounding errors. 0.1+0.2 =/= 0.3 because decimal numbers cannot reliably be represented in Os and 1s.It would be something like 0.30000000000000004 or something equally cursed.

It's a fundamental law of how binary floating point (IEEE 754) works. For 50+ years we have just ignored this because who tf cares but now with AI in the picture. You can't simply ignore it. Millions of matrix multiplications per second,millisecond inferences and perfect consistency across training runs. It means even than tiny errors get magnified into something catastrophic across 175 billion parameters.

This isn't a huge problem generally neural networks don't need mathematical perfection infact gradient descent actually loves a bit of noise and garbage for generalization but the problem is here we are dealing with algorithms. There's ofc workarounds like quantization, tensor cores in GPUs specifically designed to handle this with FP32 but there's none that specifically catering to our needs because we require a deterministic bit exact reproducibility.

idk anything about computers so ignore any following idiocy
this is floating point precision errors correct?
this is just unfixable isnt it?
is quantum computing a fad? if not would it fix this? same with tertiary computers, although unrealistic to inplement

 

[Sayori]

I'll keep it as simply as possible so anyone can understand. You see all computers store things Os and 1s. Everything is long sequences of Os and 1s but the problem is all systems since the dawn of computers have rounding errors. 0.1+0.2 =/= 0.3 because decimal numbers cannot reliably be represented in Os and 1s.It would be something like 0.30000000000000004 or something equally cursed.

It's a fundamental law of how binary floating point (IEEE 754) works. For 50+ years we have just ignored this because who tf cares but now with AI in the picture. You can't simply ignore it. Millions of matrix multiplications per second,millisecond inferences and perfect consistency across training runs. It means even than tiny errors get magnified into something catastrophic across 175 billion parameters.

This isn't a huge problem generally neural networks don't need mathematical perfection infact gradient descent actually loves a bit of noise and garbage for generalization but the problem is here we are dealing with algorithms. There's ofc workarounds like quantization, tensor cores in GPUs specifically designed to handle this with FP32 but there's none that specifically catering to our needs because we require a deterministic bit exact reproducibility.

so, since your company works with ai its caused it to stuck rn ?

Wish you luck and hope that you are a crucial member of your company as to not get fired

 

[jeoyw9192]

Always some bullshit with this old nigga

If you have nothing meaningful to say (including questions) stfu.

I'll keep it as simply as possible so anyone can understand. You see all computers store things Os and 1s. Everything is long sequences of Os and 1s but the problem is all systems since the dawn of computers have rounding errors. 0.1+0.2 =/= 0.3 because decimal numbers cannot reliably be represented in Os and 1s.It would be something like 0.30000000000000004 or something equally cursed.

It's a fundamental law of how binary floating point (IEEE 754) works. For 50+ years we have just ignored this because who tf cares but now with AI in the picture. You can't simply ignore it. Millions of matrix multiplications per second,millisecond inferences and perfect consistency across training runs. It means even than tiny errors get magnified into something catastrophic across 175 billion parameters.

This isn't a huge problem generally neural networks don't need mathematical perfection infact gradient descent actually loves a bit of noise and garbage for generalization but the problem is here we are dealing with algorithms. There's ofc workarounds like quantization, tensor cores in GPUs specifically designed to handle this with FP32 but there's none that specifically catering to our needs because we require a deterministic bit exact reproducibility.

Thing is for instance are you considering decentralized AI networks as well? Practically ud have multiple nodes that might require verification of a certain computation. In what you mention of you don't bit exact outcomes then the nodes could end up disagreeing on the valid model state regardless of the same inputs/outputs.

Another point is the issue of being able to debug at a much larger scale, like when a model training run costs a shit ton of money like in the millions , the bugs here are annoying asf to deal with. I'm talking finding the precise MS where a gradient exploded.

 

[childishkillah]

I didn't know this, what kind of IA are you working with?

 

[Jason Voorhees]

idk anything about computers so ignore any following idiocy
this is floating point precision errors correct?
this is just unfixable isnt it?
is quantum computing a fad? if not would it fix this? same with tertiary computers, although unrealistic to inplement

Quantum computing not a fad. Is revolutionary and progressing steadily for certain hard problems but won't fix this for today's Al training. Quantum bits have their own massive noise/error issues. It's a seperate rabbit hole entirely and still mostly lab experiments that are insanely hard to scale reliably

 

[Jason Voorhees]

so, since your company works with ai its caused it to stuck rn ?

Wish you luck and hope that you are a crucial member of your company as to not get fired

No one is getting fired brah. This issue isn't something specific to our company it's just end game for AI researchers. Not something actively harming us but could be huge bottle neck later.

 

[Jason Voorhees]

If you have nothing meaningful to say (including questions) stfu.

Thing is for instance are you considering decentralized AI networks as well? Practically ud have multiple nodes that might require verification of a certain computation. In what you mention of you don't bit exact outcomes then the nodes could end up disagreeing on the valid model state regardless of the same inputs/outputs.

Another point is the issue of being able to debug at a much larger scale, like when a model training run costs a shit ton of money like in the millions , the bugs here are annoying asf to deal with. I'm talking finding the precise MS where a gradient exploded.

This is actually called the determinism gap in AI.




In decentralized Al, if two nodes run the exact same input but use different GPU architectures or sometimes even different driver versions their floating point rounding will diverge. This is why the bit exact reproducible is the holy grail for researchers rn without it you can't easily verify if a node is cheating or just experiencing standard drift you u can't reliably trace when or why something like a gradient explosion happened. It makes all these million dollars AI tuning into a game of blind trial and error.

 

[birthdefect]

Quantum computing not a fad. Is revolutionary and progressing steadily for certain hard problems but won't fix this for today's Al training. Quantum bits have their own massive noise/error issues. It's a seperate rabbit hole entirely and still mostly lab experiments that are insanely hard to scale reliably

what about tertiary computers? would floating point precision errors still be an issue? hell what about analogue computers

 

[NinjaRG9]

If you have nothing meaningful to say (including questions)

I knowwhy u bulleh meh

 

[jeoyw9192]

This is actually called the determinism gap in AI.




In decentralized Al, if two nodes run the exact same input but use different GPU architectures or sometimes even different driver versions their floating point rounding will diverge. This is why the bit exact reproducible is the holy grail for researchers rn without it you can't easily verify if a node is cheating or just experiencing standard drift you u can't reliably trace when or why something like a gradient explosion happened. It makes all these million dollars AI tuning into a game of blind trial and error.

Very interesting I'll take a look; but yeah ur def right Abt it being a game of blind trial and error can't even fathom dealing with that shit

I knowwhy u bulleh meh

Wasn't trying to it's just annoying when comments like urs are made on such threads, u can always save time and choose not to send a message

 

[johnnyapple]

I'll keep it as simply as possible so anyone can understand. You see all computers store things Os and 1s. Everything is long sequences of Os and 1s but the problem is all systems since the dawn of computers have rounding errors. 0.1+0.2 =/= 0.3 because decimal numbers cannot reliably be represented in Os and 1s.It would be something like 0.30000000000000004 or something equally cursed.

It's a fundamental law of how binary floating point (IEEE 754) works. For 50+ years we have just ignored this because who tf cares but now with AI in the picture. You can't simply ignore it. Millions of matrix multiplications per second,millisecond inferences and perfect consistency across training runs. It means even than tiny errors get magnified into something catastrophic across 175 billion parameters.

This isn't a huge problem generally neural networks don't need mathematical perfection infact gradient descent actually loves a bit of noise and garbage for generalization but the problem is here we are dealing with algorithms. There's ofc workarounds like quantization, tensor cores in GPUs specifically designed to handle this with FP32 but there's none that specifically catering to our needs because we require a deterministic bit exact reproducibility.

healthcare or law? Probably already read through this but I remember thinking machines talked about deterministic inference on a single server

Defeating Nondeterminism in LLM Inference

Reproducibility is a bedrock of scientific progress. However, it’s remarkably difficult to get reproducible results out of large language models. For example, you might observe that asking ChatGPT the same question multiple times provides different results. This by itself is not surprising...

thinkingmachines.ai

 

[Jason Voorhees]

healthcare or law? Probably already read through this but I remember thinking machines talked about deterministic inference on a single server

Defeating Nondeterminism in LLM Inference

Reproducibility is a bedrock of scientific progress. However, it’s remarkably difficult to get reproducible results out of large language models. For example, you might observe that asking ChatGPT the same question multiple times provides different results. This by itself is not surprising...

thinkingmachines.ai

I've read this before. Thinking Machines approach shows that while a single GPU operation can be made deterministic, scaling that to a production is a nightmare.

 

[Jason Voorhees]

@Gomez

 

[Gomez]

@Gomez

Time to pick up that job as a carpenter
​

 

[Glorious King]

the real question is

how do you find time to do this and research more on topics like this?

istg i cant find time to read my own code base brah

@Swarthy Knight

 

[Jason Voorhees]

the real question is

how do you find time to do this and research more on topics like this?

istg i cant find time read my own code base brah

@Swarthy Knight

Subscribe to YouTube channels and tech newsletters and learn. I literally read about these things at breakfast or while running listening to podcasts

 

[Glorious King]

Subscribe to YouTube channels and tech newsletters and learn. I literally read about these things at breakfast or while running listening to podcasts

drop their names in dms pls

my feed is filled with japan slop and jdm shi

 

[Jason Voorhees]

drop their names in dms pls

my feed is filled with japan slop and jdm shi

Will dm you when I'm done shit posting in too lazy to go click share a dozen times now.

 

[johnnyapple]

I've read this before. Thinking Machines approach shows that while a single GPU operation can be made deterministic, scaling that to a production is a nightmare.

you sound AI but yeah

 

[Jason Voorhees]

you sound AI but yeah

Because it is I'm on phone. Oneplus Ai

 

[LXR]

I'll keep it as simply as possible so anyone can understand. You see all computers store things Os and 1s. Everything is long sequences of Os and 1s but the problem is all systems since the dawn of computers have rounding errors. 0.1+0.2 =/= 0.3 because decimal numbers cannot reliably be represented in Os and 1s.It would be something like 0.30000000000000004 or something equally cursed.

It's a fundamental law of how binary floating point (IEEE 754) works. For 50+ years we have just ignored this because who tf cares but now with AI in the picture. You can't simply ignore it. Millions of matrix multiplications per second,millisecond inferences and perfect consistency across training runs. It means even than tiny errors get magnified into something catastrophic across 175 billion parameters.

This isn't a huge problem generally neural networks don't need mathematical perfection infact gradient descent actually loves a bit of noise and garbage for generalization but the problem is here we are dealing with algorithms. There's ofc workarounds like quantization, tensor cores in GPUs specifically designed to handle this with FP32 but there's none that specifically catering to our needs because we require a deterministic bit exact reproducibility.

Time to prove your college tag, invent a new paradigm

 

[Jason Voorhees]

you sound AI but yeah

GReycel indian very good

 

[johnnyapple]

GReycel indian very good

I dont understand but yes

 

[topology]

I'll keep it as simply as possible so anyone can understand. You see all computers store things Os and 1s. Everything is long sequences of Os and 1s but the problem is all systems since the dawn of computers have rounding errors. 0.1+0.2 =/= 0.3 because decimal numbers cannot reliably be represented in Os and 1s.It would be something like 0.30000000000000004 or something equally cursed.

It's a fundamental law of how binary floating point (IEEE 754) works. For 50+ years we have just ignored this because who tf cares but now with AI in the picture. You can't simply ignore it. Millions of matrix multiplications per second,millisecond inferences and perfect consistency across training runs. It means even than tiny errors get magnified into something catastrophic across 175 billion parameters.

This isn't a huge problem generally neural networks don't need mathematical perfection infact gradient descent actually loves a bit of noise and garbage for generalization but the problem is here we are dealing with algorithms. There's ofc workarounds like quantization, tensor cores in GPUs specifically designed to handle this with FP32 but there's none that specifically catering to our needs because we require a deterministic bit exact reproducibility.

Do you work for a massive company? Didn't know your company specialized in AI.

 

[jeoyw9192]

Subscribe to YouTube channels and tech newsletters and learn. I literally read about these things at breakfast or while running listening to podcasts

DM me too should be helpful