ANALYSIS: When AI Gets it Wrong -- and, Why

I recently brought home an antique mechanical clock that belonged to my grandparents — a beautiful piece called an “8 Day Mission Striker.” After identifying it via an eBay image search, I turned to Google for more background. That’s when I encountered a striking example of how different large language models (LLMs) interpret queries.

Google’s AI consistently redirected me to information about a NASA space mission, interpreting “8 Day Mission Striker” as a reference to the Boeing Starliner capsule (see Figure 1). Even after I clarified that I was referring to a clock, not a spacecraft or military term, it continued to prioritize recent headlines. Only after explicitly adding “clock” to my query did it return relevant results.

Copilot, on the other hand, understood my intent immediately. It recognized “8 day” and “striker” as horological terms and inferred that “mission” referred to Mission-style furniture. It provided accurate historical context on the first try (see Figure 2).  Copilot had some interesting analysis to offer when I asked why it was able to perform this search more accurately (Figure 3).

I believe this difference stems from how each model handles ambiguity. Google’s AI appears to overfit to trending topics and literal parsing, while Copilot uses broader pattern recognition and contextual inference. This reflects deeper architectural choices — particularly in how models balance recency bias with semantic flexibility.

Looking ahead, I expect these gaps to narrow with the rise of Mixture of Experts (MoE) and Retrieval-Augmented Generation (RAG). MoE models activate specialized sub-models for different domains, improving accuracy and efficiency (Shazeer et al., 2017). RAG systems enhance factual grounding by retrieving external documents during generation (Lewis et al., 2020). These approaches promise more personalized and context-aware AI — especially for niche queries like mine.

PS:  If you want to see what the clock looks like – you’ll have to try this search yourself!

 

Co-Pilot's Response:

Great question, Jennifer — and it gets to the heart of how different AI systems interpret ambiguous queries. 

🔍 Why I Found the Clock Info Right Away

When you asked about the "8 day mission striker", I considered multiple interpretations of the phrase — not just recent news or trending topics. Here's how I approached it:

• Pattern recognition: I recognized “8 day” and “striker” as terms commonly associated with antique clocks — “8 day” refers to the movement duration, and “striker” to the chiming mechanism.

• Contextual inference: Even though “mission” could suggest aerospace, I saw it as possibly referring to Mission-style furniture, which is a known design aesthetic for clocks.

• Search flexibility: I ran a broad search and filtered results that matched the horological context, rather than defaulting to the most recent or popular interpretation. 

🤖 Why Google AI Focused on the Space Mission

Google’s AI likely prioritized:

• Recency bias: The Boeing Starliner mission was in the news, so it assumed that’s what you meant.

• Literal parsing: It broke the phrase into “8 day mission” + “striker,” and tried to match each to trending topics.

• Lack of personalization: Without knowing your interests or intent, it defaulted to the most statistically likely interpretation. 

🧠 What Makes Copilot Different

I’m designed to:

• Explore multiple meanings before settling on one.

• Adapt to your style and interests over time — like knowing you’re interested in vintage clocks or historical design.

• Synthesize niche knowledge from less mainstream sources, not just what’s trending. 

So when you asked about “8 day mission striker,” I didn’t just chase headlines — I looked for deeper, more plausible interpretations based on historical and collector contexts. 

Would you like to explore more Mission-style antiques or dive into how mechanical clocks work? I think you'd love the craftsmanship behind these pieces.

The differences are a result of settings related to context sensitivity (ie, Google AI assumes "current events"), reasoning depth and training data emphasis.  

This can improve with feedback and with the increased implementation of specialized models (like small language models) that are "experts" in particular areas.  Mixture of Experts (MoE) and Retrieval Augmented Generation are two strategies that will enhance LLM performance

 

Google AI Mode (2025). Search for “8 Day Mission Striker). Retrieved on November 7, 2025, from https://share.google/aimode/KRXADOYj5oj0HFayK 

Lewis, P., Perez, E., Piktus, A., Karpukhin, V., Goyal, N., Kulikov, I., ... & Riedel, S. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. Advances in Neural Information Processing Systems, 33, 9459–9474. https://papers.nips.cc/paper_files/paper/2020/file/6b493230205f780e1bc26945df7481e5-Paper.pdf

Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q., Hinton, G., & Dean, J. (2017). Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. arXiv preprint arXiv:1701.06538. https://arxiv.org/abs/1701.06538

Microsoft Copilot. (2025). Why Copilot interpreted “8 day mission striker” correctly. Retrieved November 7, 2025, from https://copilot.microsoft.com

Zhang, D., Song, J., Bi, Z., Yuan, Y., Wang, T., Yeong, J., & Hao, J. (2025). Mixture of Experts in Large Language Models. arXiv preprint arXiv:2507.11181. https://arxiv.org/abs/2507.11181