Uncategorized

  • AI Concept Drift and the Business/Ecosystem Lakes of Regression

    While reading the ISTQB-CTAI (Artificial Intelligence) syllabus, I came across the following text for **Concept Drift**: “The operational environment can change over time without the trained model changing correspondingly. This phenomenon is known as concept drift and typically causes the outputs of the model to become increasingly less accurate and less useful. For example, the

    Read more →

  • GPT-10: Glitches, Pranks and Tricks

    [Some time in the near future, a humanoid robot with a sleek design and expressive features, strides confidently onto the stage. The robot gives a playful wave and adjusts the microphone to its height, and awkwardly adjusts its pants over an overgrown tummy.] Hello there! Gather around, gather around. I’m GPT-10, the most advanced AI

    Read more →

  • Testing vs Tester

    Testing vs Tester

    This is in response to an important thread started by Brijesh DEB that reflects a line of thought popularised by many proficient testers out there. The post says: “Testing doesn’t assure qualityTesting doesn’t improve qualityTesting doesn’t reduce risks Testing helps you collect information and identify risks. Testing helps you discover the unknown and uncover the

    Read more →

  • Just a Drop of Bias and Ego

    I am a student of testing, a drop amongst other drops in the river of testing. Alone I am in a small water cocoon of my knowledge, ego, biases, still yearning to do better. Together, we are the river. The problem with the word ‘expert’ is that one starts thinking of self as a stone

    Read more →

  • QED: Questionably Ethical Duplicity

    Dear Testers, let’s get real for a moment. We’re living in an age where copying isn’t just a minor transgression – it’s practically a way of life. It’s like we’re all starring in the world’s longest-running, most depressing episode of “Copycat.” Take these AI language models, for example. They’ve got this photographic memory, right? But

    Read more →

  • LLMs and Negations

    LLMs and Negations

    Tomorrow I’ve a talk where as the first demo I wanted to demonstrate issues with LLMs before showing what can still be achieved. I realised that the famous: “Create the image of a room without a tiger.” does not work any more in GPT-4o, wherein it used to always include a tiger. LLMs don’t handle

    Read more →

  • LLMs: On Soundness and Completeness of a Probabilistic Test Oracle

    Imagine we have a Large Language Model (LLM) that generates responses to prompts. We also have a perfect ground truth oracle G, which always knows the right answer. Probabilistic SoundnessWe want to make sure that if our test oracle says the LLM’s answer is correct, it’s actually correct most of the time (better than a

    Read more →

  • A Name by Any Other Name Can’t Change the Game

    The only path I have cared about is the path of experimenting and gaining knowledge. Giving cute names to “manual testing” by itself does not solve the core of the problem – the problem that testing done in this manner is not just thought of as “less technical” than the fancy sounding counter-parts; it is

    Read more →

  • Some Prompting Heuristics – Making LLMs Work For You

    Following is a list of some heuristics for a better experience while prompting an LLM. The mileage might vary, we are not talking exact science here. I am going to type what comes to my mind, as I don’t have a fixed goal of coming up with THE X-number of suggestions which can change the

    Read more →

  • I wrote poetry after a long time and it is inspired by recent discussions and observations I had in the sphere of LLMs. This poetry form is called “Tappa” or “Maahiya”. This was recently seen in the Bollywood movie Animal, interestingly choreographed with a fight sequence 🙂. But it’s more of a folk form from

    Read more →