All of these resources on agentic workflows, managing agent memory, harness engineering, etc. appear to just be theatre to me.
Those were mostly obviated by reasoning models and harness updates by 2024.
It seems pointless to invest energy into the latest/greatest AI technique or framework when they're going to either be absorbed or replaced on a 3 month cycle.
These tools work pretty well out-of-the-box. I'm sure I could squeeze out better token usage or streamline some tool calls, but it's not something I really want to focus on. Just like I don't want to endlessly configure my IDE, I don't really have patience with spending time on anything besides actually building something.
LLMs seem terrible at using LLMs in harnesses. Have you seen how they rot their context with the stuff they put in .md files if you let them?
You'd have to have the LLMs search, and thus these resources could be for them more than you
Another thing in it is a strict line count. Any increase in line count requires my approval. That last one is important because it plays well with two biases: models don't tend to create long lines so they won't try to cheat that way, and they're strongly inclined to keep churning out lines so I take that away from them.
Unless I'm doing something super complicated even taking time to set things up like subagents, etc. seems like a waste compared to just building.
The only things that really seem beneficial (for Claude Code) seems to be learning to set up loops, memory and finding relevant MCP servers.
First you have to know that "it" exists and is possible. A cookbook like this introduces readers to concepts and features they didn't know were there in the first place.
Just use the vanilla settings.
[deleted]
I appreciate there may be more efficient ways to do things if you're an actual developer or engineer but I'm not and the tools I've been able to build so far have been both fun to make and add value to our workflow as a small 2 person ecommerce business.
I do at least have a background in web as a designer for many years and having worked with developers I can at least spec and understand how a product might work which is one thing Claude/AI isn't hugely helpful with and often it's the testing and QA phase as always where the problems and shortcomings expose themselves.
Now process engineering is more important than prompt engineering.
the point of guides is to provide assurance for people unfamiliar to the process in the first place
Before /goal was ralph-wiggum it was def an interesting learning experience, it was interesting to see how Claude became a lot better doing this itself but it still took 6 Month and more.
You can wait and sit it out and suddenly you get fired if you miss the point when to start spending more time and energy on topics like this.
[dead]
https://platform.claude.com/cookbook/coding-prompting-for-fr...
Before: bland
After: bland with gradients
I've also found that they tend to coerce the programmer into thinking about the end result, rather than try to push them out of that role. His grill skills are about making sure the actual requirements are known, and his prototype skill can help explore things that need to be experienced to make a hard decision on.
Other AI labs also tend to publish examples and cookbooks on GitHub and Hugging Face, so it's always worth keeping an eye on those as well.
It turns out the "average" version of a recipe that's baked into the weights is usually a solid recipe, and they're really good at offering substitutions for things like "I don't have ingredient X" or "make it vegetarian".
It's also fun promoting "make it tastier" once or twice after each recipe just to see what happens.
I'm not enough of a designer to be able to point exactly at what makes it so, but Claude does seem to have a somewhat limited repertoire of styles.
Maybe if you could point more precisely at the required changes, you could discourage it?
[deleted]
I don't need to tell the agent my tech stack, what DB access library I'm using, the way in which I write my tests, etc. Why? Because it's all very clearly spelled out within the code itself. I've gone great lengths to make sure that my fairly complex domain can be understood by a person with little domain knowledge, which means it should certainly be understood by a tool that has a wealth of it.
I can't use it on good faith Same goes for grok, chat gpt and others all involved in killing people and monopolising the market.
OpenAI's started in June 2022 https://github.com/openai/openai-cookbook/commit/535f545be7e...
[deleted]
[dead]
[dead]
[dead]
Of course, there are other areas that can improve model output (Direction rather than open-ended assistance requests, using keywords + plugins that help, the "your output should include: " style prompting).
A few of us run almost the same exact setup at my shop (Base Claude Code w/ SuperPowers + a context repository) and the models are somewhat unhelpful to some, and give meaningful output to others. The only correlation I notice is that their prompts are no-good. Not from a meta "prompt" engineering standpoint, but from a general English 101 standpoint.
"dudde no i wanted the function to return 3 things. not like that. do it again"
VS something like
"Modify the "renderThreeVars()" function signature to accept another variable called "z" and add it to the return statement at line 64."
Obvious exaggeration, but you get the point.
I ask it all the time about whether X is feasible, how we can get started on Y, and to investigate issue Z.
It is working great for me in a >100k LOC project.
Perhaps this works less well with weaker models. I suspect the people who say Qwen 3.6 27B is working well, are using prompts like "modify the renderThreeVars() function in rendering.py".
The irony to me is that a lot of what I'd tell people about this is exactly what I would have told them about writing Stack Overflow questions.
For example, if I tell it that I want my app translated, it can plan for me what the recommended options are in my framework, what languages I should target for my app, and come up with a skill for a repeatable workflow.
[deleted]
I've noticed among my coworkers that we all have different amounts of trust we're willing to give the agent. That seems to manifest into some people only asking questions about the existing code, but never writing anything new with it. Others are willing to do limited targeted changes with the agent, but are unwilling to do things like let it make commits, or connect MCP servers, or really do anything that isnt fully understood by the human before setting the agent loose. Then I find myself, who has dove headlong into it all. I have skills that use MCP servers to check for pull requests, and give me summaries to give me more context for code reviews. I update Jira tickets in batches of 50+. I develop complete features exclusively through prompting the agent to do everything. I know I still have the responsibility to understand it at the same level as if I wrote it my self, and defend it and debug it.
I can easily see that superpowers would be wasted on most of my coworkers, simply because the benefits compound with the complexity of my ask. My coworkers aren't willing to hand off enough control to receive the benefits of superpowers.
From some very quick tests, I get the impression that having good grammar, punctuation etc. is really not important (although I try to do it anyway because I'm accustomed to trying to do it), but clarity and precision definitely are. And of course, if you have unknown unknowns, they do need to get figured out before progress is possible. But with the right mindset, that just means you need an extra turn or two, not that you're going to end up at a dead end. (... I guess that counts as a pun?)
The prompts were, predictably, really bad. Broken English, sentence fragments, vague requests, lack of context. Yet somehow, the users always got the answer they were looking for. It might have taken a few extra turns with questions from the model, but the end result was the same.
It's humbling, but a flowery, carefully crafted prompt is at best slightly more efficient than a "CAN A DOG BE EATIN SUN FLOWER SEED?" peasant prompt.
Unless the user wanted to know if a cat could eat sunflower seed or something.
ChatGPT kept the charade for all of one sentence. Then it dropped to talking about "your dog" the rest of the way. It even starts the final paragraph with "if, instead, you mean you (a human) ate them...", and finishes with the question "is this about an actual dog or yourself?"
Google did consistently refer to me as a dog, but its entire focus was on the steps "your human" should take, no advice for the dog itself.
In both cases, it looks like the first person roleplay was entirely inconsequential for the usefulness of the output. I think the current gen AIs have outgrown this trick and you can safely forget about it.
They don't need to be told that you're asking about the safety of eating them, because they can infer that based on the fact that a very large percentage of any text linking dogs to sunflower seeds is obviously going to be about the safety of the dog eating them.
Even pre-LLM that would have been a perfectly sufficient google search for the same information.
Why is would that be "bad prompt"? It is machine inpit, if machine can interpret fragment all the better.
Are people really putting on their resumes that they are capable of reading and writing and appropriately defining and limiting context? That's all prompt engineering is - it's being able to communicate effectively and elucidate your objectives.
Congratulations to all you English majors out there, you're about to make $350K/year.
People put whatever buzzwords will get them through initial screening. My resume contains tons of banal shit like agile, automated testing, Linux, AI (since 2018), and design patterns.
Sure, but I think it's essentially just that people who are better at traditional non-agentic software engineering are better at agentic software engineering. The only exception would be individuals who avoid agentic coding due to skepticism, hostility, or lack of opportunity.
[dead]
Yes, it is. Source: had models inplement complex things from scratch and bullshit regardless of whether it was a one-line prompt or a detailed "SOTA witchcraft magic spells that are guaranteed to work"
It's like imagining you could "prompt" Richard Feynman to be smarter at Physics.
That is, for 99.9% of engineers, if you want the model to do a code review of your project, the best solution is to just ask Fable, "Hey Fable, do a code review of this project." Throwing in extra text like "think like a senior engineer", "ensure you focus on DRY principles, KISS, self documenting code, etc", doesn't make a difference.
These sorts of tricks used to work with dumber models, but now, like I said before, it's like thinking you can prompt Linus Torvalds into writing better C than he already can do.
> it's like thinking you can prompt Linus Torvalds into writing better C++ than he already can do.
Linus Torvalds, the inventor (and beloved dictator) of Linux, has always been quite harsh about C++ and why he rejects it for Linux kernel development. He’s not just been very vocal about it, but also brought up some arguments against the use of C++ that are worth reviewing in detail.
The models are beyond expert level in many areas at this point.
Do you really believe that adding extra junk to your prompt is going to make the model write code better than it does already?
Again, imagine going to Terrence Tao and "prompting" him to get better at Maths, do you think you can do it? What prompt would you give to him to make him produce better maths. Unless you're already a world-leading Mathematician I think you would find it hard.
[deleted]
[dead]
Being able to write clear, always is.
Prompt engineering was a GPT-3 era term, which couldn't understand instructions. Then ChatGPT came out in late 2022, which made actual prompt engineering superfluous.
For example ”come on” and then the model will just do the reasonable thing instead of disobedience. I imagine that ” broooo pleaseeee!” would work just as well.
Though sometimes I just clear the context or fork from the past, but this might or might not give any better results.
At least cognitive load is higher for me especially with forking so I prefer not to do it
If the only artifacts remaining on earth a million years past our demise were The Parthenon and 2002 keygens, our existence will have been worth it.
"Use a black background and this hideously wide font!"
[1]: https://github.com/anthropics/claude-code/blob/main/plugins/...
Personally I prefer the before shots all the way down.
I have been using Magic Patterns which is very good at generating initial prototypes, especially when driven by an agent familiar with it's facilities and constraints.
I assume the service is just a wrapper plus scaffolding but I like it. You have to push it though to get anything particularly creative but it's good at standard front end designs.
https://www.magicpatterns.com/
That's not necessarily what you're after because I only use it for piloting stuff, not for adhoc feature addition and fixes.
We resolved the issue and Claude chirpily commented "So breakfast just became brunch".
One of my favorite features has been giving a recipe url and having it figure out the calories for that recipe with line-by-line entries for each ingredient. Then it keeps the extracted recipe in markdown format for the next time I want to make it.
Substitutions are also great - "I'm making [such and such recipe], but with half the sugar."
One of the others has been "I had a meal a couple weeks ago at [x restaurant], can you enter that again for right now?" Having the long-term context for entry has been excellent. And a related one, where I upload a photo of the menu, and say which menu items I'm eating and it puts together an estimate for the calorie count.
I lean heavily on a long auto-compacting claude code session for all of this. I have a UI for graphs, history, and simple entries, but claude's been my favorite means of entry by far
Works surprisingly well, can even tell it "I have so and so ingredients in my pantry, I want a keto-friendly meal, what can I make?" followed with some narrowing down dialogue-style.
You're probably burning tokens. A markdown file with a summary, structure of your project, rules, and even goals prevents Claude from hunting for and reading files over and over. Each new session (or after compact) would be starting from scratch. I take it a step further and create a HANDOFF.md file between compacting that gives the next session full context of what was done and how to continue.
Also:
"Treat CLAUDE.md as the place you write down what you’d otherwise re-explain. Add to it when:
Claude makes the same mistake a second time
A code review catches something Claude should have known about this codebase
You type the same correction or clarification into chat that you typed last session"
This is to say, Anthropic seems to be going down the path you are describing to make the CLAUDE.MD unnecessary. But I think that's the Claude Code harness.
If you're just having an agent produce code, then I'd agree. I'd argue that any additional context it might need ought to be included in README.md anyways, just like a human.
But I have projects where agents do more than just code, and retaining context about how I want the agents to operate, keeping track of gotchas, avoiding unnecessary steps in the future, etc., just ends up going into CLAUDE.md. It's not for me; it's for the agents. I also never edit this file directly. I may review it every once in a while to make sure there isn't anything weird in there, but I leave it to the agent to add what it thinks is necessary (or I'll tell it that something will be relevant in the future and that it should add it, etc.).
Although even this pattern isn't necessarily future-proof. Seems like skills are kind of taking that role. Still, I end up with a CLAUDE.md in these kinds of scenarios to catch all the little pieces of context that don't fit nicely anywhere else.
That's definitely a downside of a minimal MD and I could def be wasting tokens. I've tried to scope the tasks such that it won't feel the need to scan unnecessary files, but it's not bulletproof.
My system CLAUDE.MD spells out my preferences in stack and architecture, and how I expect the agent to interact with me.
I have it generate a summary doc in large repos so it doesn't need to slurp in the world to review a PR or work on a corner of it.
Been working on some things that require more targeted, smaller scale changes and the base models do perfectly fine when given good instructions.
Superpowers and Compound Engineering are the two that I hear about around our team, both seem "fine" if you're into the fully agentic engineering "big change" future technology stuff.
Extra prompt text isn’t to make the model smarter, it’s to pull attention towards what you want. “Do code review” is far different from “Make sure changes align with existing architecture” or “follow these enterprise standards XYZ”. The model just acts as an average of its training data, which may or may not align with your goals.
It's not that the models aren't smart or whatever; we know they're extremely capable.
There's always going to be value in being able to clearly communicate what you want the model to do, especially when the context is lacking.
[dead]