When life gives you LLMs make LLMonade