Ben Auffarth

🌟 Exciting blog post from NHS England on Reproducible Analytical Pipelines  🥁 Keep banging the drum Sam Hollings 📈 NHS England are sharing their journey on Reproducible Analytical Pipelines (RAP) and how it's enhancing the quality of their data processes. 🔍 What is RAP? RAP aims to automate statistical and analytical processes, making data analysis more efficient, transparent, and robust. This means less manual work and more time for data analysts to focus on what truly matters - delivering insights that improve healthcare services. 🌐 Why It Matters for the Culture of Data Services? Adopting RAP aligns the commitment to enhancing the quality and reliability of data services. It's not just about technology; it's about fostering a culture where continuous improvement and collaboration are at the forefront. 🛠️ Progress NHS England Data teams are integrating RAP across the organisation and It's still early days, but the impact is tangible and transferable. 📈 Looking Ahead They are committed to expanding the use of RAP, enhancing data capabilities, and continuing to share their progress. Hoping to hear more of this type of work. #NHS #DataScience #Innovation #RAP #HealthcareTechnology #Teamwork #Data #Health #Collaboration Link to the source in the comments. Image created by DALLE-3

17

1 Comment

SparseEmbed. This work combines the strengths of sparse (SPLADE) and dense (ColBERT) representations for first-stage retrieval. TL;DR: - For a given sentence, sparse vectors are generated (similar to SPLADE) - For the same sentence, contextualized embeddings are generated for each token (similar to ColBERT). However, the attention layer only uses the logits of the sparse vectors (including token expansions) to generate the embeddings. - This results in faster training and smaller contextualized embeddings for each token. - Retrieval is hybrid. First, keyword matching matches the query to document expanded tokens. Then, scoring is computed based on the dot products between query and document contextual embeddings.

47

RAG using tabular data with function calling - my talk from @pydatalondon is now available on youtube: The talk is scattered with references to Monty Python's Quest for the Holy Grail (tabular data is the holy grail of RAG for Enterprises). https://lnkd.in/dAkSUMiE

13

🌟Announcing ROLEBENCH, a framework designed to evaluate the performance of Role-Prompting across various datasets and Large Language Models (LLMs). The results across some benchmark datasets demonstrates better performance than results mentioned in model cards [Refer to Github Repository for results]. A free-tier colab notebook to test ROLEBENCH on Llama 3-8B: https://lnkd.in/g4zDRHZf Role-Prompting assigns specific roles to the AI, such as acting as a teacher, a historian, or a technical expert, to see how well the model can adapt its responses based on these roles. ROLEBENCH assess how well different LLMs perform when given these role-specific prompts. This tool will offer valuable insights into the capabilities and limitations of current LLMs. ROLEBENCH currently supports Llama 3-8B, Phi-3-mini, Mistral-7B and Gemma-7B across some benchmark datasets. Github : https://lnkd.in/gbsax-hX #AI #MachineLearning #RolePrompting #AIResearch #LLM #DataScience

80

4 Comments

I did not know what Noise Contrastive Estimation was, and now I do. Its neat. The Compass blog is such an accessible way to learn about what is hot in machine learning and computational statistics!

5

Ever wondered about the significance of random.seed(42) in Python? It's more than just a number – it's a pop culture Easter egg! The choice of 42 as a seed is a nod to Douglas Adams' The Hitchhiker's Guide to the Galaxy, where it's famously the Answer to the Ultimate Question of Life, the Universe, and Everything. While any integer works as a seed, using 42 is a playful wink to fellow programmers and sci-fi enthusiasts. It doesn't affect the randomness, but it does add a dash of humor to your code! 😄 Next time you see random.seed(42), you'll know you're in on the joke. Happy coding. Read more... #Python #Programming #PopCulture #CodingHumor #DataScience

1

🔍 Google's Gemma Scope: Illuminating the Inner Workings of Large Language Models 🧠 Google researchers have developed a powerful new tool called Gemma Scope, designed to shed light on how each layer in Gemma 2 large language models responds to input tokens. 📊 Key insights: - Sparse autoencoders (SAEs) can transform embeddings into interpretable representations - Each index in the transformed embedding corresponds to a distinct concept - SAE weights indicate the strength of each concept in the input Gemma Scope enables: - Manual and automatic labeling of concepts in each layer - Steering the model by adjusting SAE outputs to generate concept-specific text This groundbreaking tool paves the way for answering critical questions about LLMs, such as how fine-tuning and chain-of-thought prompting influence a model's internal representations. 🤔 #GemmaScope #LargeLanugageModels #TransformerInterpretability #GoogleAI #MachineLearningResearch

1

An easy, intuitive video to understand RESNET. Throughout my career I find the coolest thing about maths is that it all come back to fundamental ideas. Neural Nets are just approximation functions RESNET has an equivalence in Numerical Integration as a EULER step https://lnkd.in/gyNidB5A

4

How Much Research Is Being Written by Large Language Models? 17.5% (that is assumed to have been detected) https://buff.ly/3VdqVv0 This will mean all future papers on average will be referencing 3 to 5 papers containing LLM generated content significant error rates.

1

An interesting post on how Skyscanner uses the combination of Dr Jekyll and its in-house statistical tool called WISE to ship experiment variants to production. Instead of using a frequentist approach as is done in most AB experiments, this framework uses a Bayesian approach to quantify the uncertainty in the final estimates by reporting credible intervals instead of confidence intervals. Summary: - Example of an experiment to test whether a pop-up, mentioning that ‘Skyscanner never takes a cut’ increases the ‘redirector rate’ or not. - If sufficient data has not yet been collected, WISE’s results will say ‘keep testing’. - WISE is able to provide a recommendation on the variant to ship. - WISE uses the Bayesian approach. They model their belief of the underlying parameter values as probability distributions, which are updated as new data is observed. - For conversion metrics, it uses a beta-binomial model - For revenue per user metrics, it uses a hurdle gamma-exponential model - What's the experiment stopping criteria: Through Monte Carlo simulations of thousands of experiments, they have determined a set of bespoke target 90% HDI widths to serve as their stopping criterion, which ensured an appropriate balance between the duration of experiments and the correct decision rate. https://lnkd.in/dv7BEHXe #statistics #analytics #rstats #python #peopleanalytics #experimentation

6

1 Comment

A splendid scaling, denoising, 🗑️ in 🗑️ out analysis for LLM. It also clarifies why we (non giant tech) don't train base models. Must read (we want more of this 😄) https://lnkd.in/extYPSax

8

Over the holiday weekend I came across an insightful article [https://lnkd.in/eeQ6pNAT] by Julian Flaks of EQengineered on the importance of providing due consideration to data quality as data-driven software is developed. It's great to see more thought leaders in our industry emphasizing this critical aspect of data management. In my recent article, I explored similar themes, drawing parallels between data and a well-maintained car, highlighting the necessity of regular maintenance for optimal performance. If you're interested in diving deeper into how proactive data care can drive efficiency and reliability, check out my piece here: https://lnkd.in/eE6Nr4Hy Kudos to Julian for contributing to this important conversation. Let's continue to prioritize data health and drive our industry forward! 🚀 #DataManagement #DataQuality #PreventativeMaintenance #DataCare #TechTrends

4

Synthetic data generation has become a huge topic in the last few years. As with many new and evolving technologies, there are still no standard evaluation techniques. I have put my thoughts on this into this article … #SyntheticData

6

1 Comment

Really happy to share that our work on “Learning Metrics that Maximise Power for Accelerated A/B-tests” will appear in the Applied Data Science track of #KDD2024. We address a common problem in industry: how can we minimise type-II errors in online controlled experiments? Can we simply learn a metric combination that directly minimises this quantity? This project started out as “let’s implement the ideas from this great paper by Eugene Kharitonov et al.”, but we ended up proposing some extensions to their seminal work that seem to be quite effective, and are now used across ShareChat. Proud to have this work out there, let us know what you think. Additionally, we were assigned an Area Chair that clearly cared: initial low-confidence reviews prompted them to invite more reviewers (we got 8!), and the AC themselves correctly grilled our rebuttals to point out arguments they deemed insufficient. It’s easy to be positive about peer review when your paper gets accepted, but it seems there are still some unsung heroes out there... Work with Aleksei U.. ACM SIGKDD & Annual KDD Conference

116

9 Comments

In computational linguistics, researchers are focused on enhancing language models' ability to extract precise information from extensive textual data. Recent studies introduce "retrieval heads," specialized attention mechanisms that selectively focus on critical segments of large texts, improving targeted data retrieval. These heads, integrated into transformer-based models like LLaMA and Yi, significantly enhance accuracy and efficiency, particularly in handling long-context scenarios. The methodology involves thorough experiments across various models, including the Needle-in-a-Haystack test, which embeds specific information to measure precision. Results show models with active retrieval heads outperform those without, with accuracy dropping notably when these heads are masked. The empirical evidence underscores the effectiveness of retrieval heads in improving information retrieval precision and reliability. This research deepens our understanding of attention mechanisms in large-scale text processing, paving the way for more efficient and accurate language models. https://lnkd.in/dNNPvMXw https://lnkd.in/ddbHX6EA Innovation Hacks AI #aiinnovation #generativeai

1

Synthetic data generation (and data augmentation in general) is a relatively underexplored area in time series. It really does wonders in the context of pretrained time series models. That said, what's currently missing from the literature are ways to quantify the "goodness" and "diversity" of synthetic data without evaluating the models on downstream tasks.

27

Kolmogorov–Arnold Networks seem like a paradigm shift in ML architectures. Curious to see how they evolve. "While MLPs have fixed activation functions on nodes ("neurons"), KANs have learnable activation functions on edges ("weights"). KANs have no linear weights at all -- every weight parameter is replaced by a univariate function parametrized as a spline. " https://lnkd.in/gdPk_qCQ

29