[email protected]

Par vienu no nozīmīgākajiem sasniegumiem zinātnē 2024. gadā @LZA_LV atzinusi pētījumu "Robota kognitīvā uztvere un augsta līmeņa instrukciju interpretācija ar dabiskās valodas jēdzieniem". Lepojamies ar mūsu @imcs_dl komandas ieguldījumu tā īstenošanā sadarbībā ar @edi_riga.

OpenAI staff comments on o3 thread. Public bookmarking 👇

We're releasing a preview of OpenAI o1—a new series of AI models designed to spend more time thinking before they respond. These models can reason through complex tasks and solve harder problems than previous models in science, coding, and math. openai.com/index/introducing…

Šonedēļ pasauli pāršalca kārtējā AI ažiotāza: @xai izlaida lielāko atvērto #LLM - x.ai/blog/grok-os Pateicoties #ValoduTehnoloģijuIniciatīva superdatoram, @AiLab_lv izdevās milzu #Grok iedarbināt un pārliecināties par tā spējām saprast un ģenerēt tekstu latviešu valodā.

In case you were wondering how superconductors work, this is a good explanation. They are a very interesting phenomenon and may prove economically useful, but are not a hard requirement for a sustainable energy future. Physics has many powerful tools of reasoning to understand and predict reality. Those tools, like first principles analysis and thinking in the limit, are broadly applicable to anything, in my experience. With Grok, @xAI is attempting to create an AI that reasons from first principles, which is fundamental if you care about getting as close to the truth as possible. The acid test would be reaching a conclusion that is correct even if it is extremely unpopular, which means being right even when the training data is almost entirely wrong. For example, Galileo concluded, after observing the moons of Jupiter from a telescope he engineered, that it was far more probable that Earth revolved around the Sun than the other way around. This view was so unpopular that he was forced to recant and placed under house arrest! If you had trained an LLM on material back then, it would’ve given you the popular, but wrong, explanation. Due to social and legal pressure, it likely wouldn’t even acknowledge the possibility that the Earth revolved around the Sun. For AI to help us understand the true nature of the universe, it must be able to discard the popular, but wrong, in favor of the unpopular, but right.

Human = Dog + ChatGPT

Mistral AI releases first ever OpenSource MoE model Fun facts: - Released as 87 GB torrent - Seems to be a scaled down version of GPT-4 - Released on X, no press release and refuse to elaborate Google left the AI community in awe with the over rehearsed demo video that's now getting overly criticised. While the Opensource AI startup Mistral AI released a MoE model consisting of 8 7B experts. What's Mixture of Experts (MoE)? Mixture of Experts (MoE) is a technique used in LLMs to enhance their efficiency and accuracy. This approach works by dividing complex tasks into smaller, more manageable sub-tasks, each handled by a specialized mini-model or "expert". Here's a concise breakdown: 1. Expert Layers: These are smaller neural networks trained to be highly skilled in specific areas. Each expert processes the same input but in a way that aligns with its unique specialization. 2. Gating Network: This is the decision-maker of the MoE architecture. It evaluates which expert is best suited for a given piece of input data. The network calculates a compatibility score between the input and each expert, and then uses these scores to determine the involvement of each expert in the task. Together, these components ensure that the right expert handles the right task. The gating network effectively routes each input to the most appropriate expert(s), while the experts focus on their specific areas of strength. This collaborative training leads to a more versatile and capable overall model. Details around Mistral's new MoE (from Reddit) For the inference of each token, only 2 experts are used. This information can be derived from the model metadata: {"dim": 4096, "n_layers": 32, "head_dim": 128, "hidden_dim": 14336, "n_heads": 32, "n_kv_heads": 8, "norm_eps": 1e-05, "vocab_size": 32000, "moe": {"num_experts_per_tok": 2, "num_experts": 8} Comparison with GPT-4 Mistral 8x7B uses a very similar architecture to that of GPT-4, but scaled down: - 8 total experts instead of 16 (2x reduction) - 7B parameters per expert instead of 166B (24x reduction) - 42B total parameters (estimated) instead of 1.8T (42x reduction) - Same 32K context as the original GPT-4

A short intro to the the Deep Learning course in University of Latvia youtu.be/2K2RMy-5ZsM

Can machine learning explain the world at the quantum level? New in @ScienceMagazine, with the code also released, our team shows that neural networks can radically improve density functionals with #DM21. Read more: dpmd.ai/density-functionals Code: dpmd.ai/density-functionals-… 1/

No industry can ignore the competitive differentiation and value that AI will unlock. @AndrewYNg from @landingAI sees 2022 as the year AI moves from big data to good data. McRock's Fantastic Fourteen mcrockcapital.com/fantastic-… #ai

Among public universities worldwide, Canadian professor salaries are the highest. (All Canadian universities are public.)

As many other works in the literature, Discrete valued neural communication was an attempt to understand this problem. arxiv.org/abs/2107.02367

28. un 29.06., Tallinā norisināsies 17. Baltijas Intelektuālās sadarbības konference par tēmu “Matemātika sabiedrībai” (Mathematics for Society). Sanāksmei aicinām sekot #Estonian #Academy of Sciences mājaslapā: akadeemia.ee/en/events/the-1…