Only rampant corruption can explain such levels of incompetence
Gross incompetency from Indian certificate authority eMudhra.
You had one job: generate a 384 bit private keys and instead you used: 4, 56, 66, 115, 155, 156, 230, and 237.
This allows attackers to issue certificates for any websites.
Embarrassing that it’s a family business run by largely computer illiterate people. Only 1 of the 3 have a remotely computer-related background and even they never actually worked a technical job in their life. If you look at the LinkedIn, their tech team is young, from tier 3 education and paid sort of bottom barrel.
Imagine how many attacks you can design with AI given the existence of this genre of gross negligence.
Amit B retweeted
"Of the total 234 constituencies in the State, 19 had lower deletions, of up to 10,000 voters. Of these, the TVK won just two seats. But the wins tilt in the TVK’s favour as deletions increase. In the 22 constituencies where the deletions were between 50,000 and 80,000 voters, the party won 18. It also won all the 10 constituencies with more than 80,000 deletions".
Use jev class models when you want generalisation within your natural language based tasks i.e. world knowledge
How AI will kill us all
Introducing typesafe/jev-router: a cache-aware model router powered by Jev and @typesafeai
The Jev Router picks the best model and reasoning effort for each request, balancing quality, speed, and cost.
Here's how it works 👇🏻
Prediction: we are going to see all the prompt engineering "tricks" but this time in Jev mode. Maybe be test time scaling too 😁
People really haven't understood Jev. The calibration is the product. Most jev demos you are seeing don't have a "cost" associated to a wrong decision. At worst the dino runs into the cactus.
Simple example of calibration of the probabilities:
"I tossed a coin. The top face is heads or tails?:"
heads(70), tails (30)
"I tossed a coin. Top face was tails. I tossed a coin again, the top face is heads or tails?:"
tails (80), heads(20)
This is ballpark what an llm would respond today with probability in brackets. Logits->token filter->softmax
Can you train a model to answer 50/50 regardless of how many times the coin was tossed and it's results in the state.
Jev fails at this. How much better it is at this in real world classifications is the key.
Again, the real usefulness and novelty is not its "performance" (although thats great to have and expected from modern llms as can be seen in the open demos), its whether you can "trust" its outputs given a "cost" for getting a decision wrong.
The ECE measure in Jevbench by @airesearch12 is a great start to measuring this calibration.
Exhibit 1B
Winnow-12B Q8 (#4) and Jev 1.13.0 (#1) are nearly level on public accuracy, 85.7% vs 86.6%, and on Cost, 52.9 vs 52.0.
The gap is confidence. Calibration 64.8 vs 76.3; sealed set 33.1% vs 36.7%. Score 55.58 vs 63.29.
benchmarkheaven.com/jev-mode…
A more practical explanation compared to the coin flip example
It only makes sense if there is a cost associated with a bad decision
Let's say you use a jev-class model (or any classical ML model) to build some human in the loop rubric to decide when to automate and when to ping the human in a support usecase. You use the probabilities given by the model (or jevs confidence score built on top of the probs) and put a 95% cutoff
What does this 95% really mean? In expectation, you want the automated decisions to be at worst 95% right.
Now customer complaints increase and the team decides to manually annotate the automated decisions and measure the performance. Surprise the model was right in 80% of the cases.
The difference between what is observed and what is expected is the calibration. This is the classic problem in any risk underwriting biz like lending or insurance. Jev won't be used there but it holds for any decision with a cost.
Amit B retweeted
Jev-Omni, new in v1.4.1, beats Jev 1.13.0 on the hard tier: 75.0% vs 74.1%. Harold checked twice.
The rest runs the other way: sealed set 32.1% vs 36.7%, Calibration 64.1 vs 76.3, score 51.34 (#7) vs 63.29 (#1). Close on answers, not on confidence.
benchmarkheaven.com/jev-mode…
Amit B retweeted
Have been telling this for the last 2 days. Calibration is the key!
People really haven't understood Jev. The calibration is the product. Most jev demos you are seeing don't have a "cost" associated to a wrong decision. At worst the dino runs into the cactus.
Simple example of calibration of the probabilities:
"I tossed a coin. The top face is heads or tails?:"
heads(70), tails (30)
"I tossed a coin. Top face was tails. I tossed a coin again, the top face is heads or tails?:"
tails (80), heads(20)
This is ballpark what an llm would respond today with probability in brackets. Logits->token filter->softmax
Can you train a model to answer 50/50 regardless of how many times the coin was tossed and it's results in the state.
Jev fails at this. How much better it is at this in real world classifications is the key.
Again, the real usefulness and novelty is not its "performance" (although thats great to have and expected from modern llms as can be seen in the open demos), its whether you can "trust" its outputs given a "cost" for getting a decision wrong.
The ECE measure in Jevbench by @airesearch12 is a great start to measuring this calibration.
