LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworks

at generating plausible solutions, but they have no guarantees whatsoever. So you need to pair them with a verifying system. This can be unit tests, integration tests, static / type checks, formal methods, etc. The point is that if you don't have these "verifier" systems you are creating an open loop and your code will quickly devolve to nonsense [0]. In my view, the best existing languages for vibe coding are: - Rust: reasonably popular, very powerful and strict type system, excellent compiler error messages. If it compiles you can be confident that a whole class of errors won't exist in your program. Best for "serious" programs, but probably requires more back and forths with the coding agent. - TypeScript: extremely popular, powerful type system, ubiquitous. Best for rapid iteration. - Luau: acceptably popular, but typed and embeddable. Best as a real-time scripting sandbox for LLMs (like our use case). I think there is space for a "Vibe-Oriented Programming" language (VOP as the author says), but I think it will require the dust to settle a bit on the LLM capabilities to understand how much can we sacrifice from the language's lack of popularity (since its new!) and the verifiability that we should endow it with. My bet is that something like AssemblyScript would be the way to go, ie, something very, very similar to an existing, typed popular language (TS) but with extra features that serve the VOP needs. Another aspect to consider besides verifiability is being able to incrementally analyze code. For structured outputs, we can generate guaranteed structures thanks to grammar-based sampling. There are papers studying how to use LSPs to guide LLM outputs at the token level [1] . We can imagine analyzers that also provide context as needed based on what the LLM is doing, for example there was this recent project that could trace all upstream and downstream information flow in a program thanks to Rust's ownership features [2]. Finally, the importance of a LLM-coding friendly sandbox will only increase: we already are seeing Anthropic move towards using LLMs to generate script as a way to make tool calls rather than calling tools directly. And we know that verifiable outputs are easier to hillclimb. So coding will get increasingly better and probably mediate everything these agents do. I think this is why Anthropic bought Bun. [0] very much in the spirit of the LLM-Modulo framework: https://arxiv.org/pdf/2402.01817 [1] https://proceedings.neurips.cc/paper_files/paper/2023/file/6... [2] https://cel.cs.brown.edu/paper/modular-information-flow-owne...

across planning and puzzle tasks [2]. In other words, verification is *harder* than generation for today's autoregressive models, so you need a checker that actually reasons about the world (compiler, linter, SAT solver, ground-truth dataset, etc.). Because of that asymmetry, stacking multiple LLMs rarely helps. The "LLM-Modulo" position paper argues that auto-regressive models simply can't do self-verification or long-horizon planning on their own and should instead be treated as high-recall idea generators wrapped by a single, sound verifier [3]. In my tests, replacing a five-model "debate" with one strong model + verifier gives equal or better answers with far less latency and orchestration overhead. [1] https://www.youtube.com/watch?v=0u2hdSpNS2o - (How) Do LLMs Reason/Plan? (talk at Microsoft Research, 11 Apr 2025) [2] https://arxiv.org/abs/2402.08115 [3] https://arxiv.org/abs/2402.01817 (related to the talk in #1)

indeterministic/backtracking planning routine. Using additional training and/or in-context techniques and prompt reformulation can improve the result further. Since the action predicates use retract/assert primitives to change state, Quantum Prolog's vardb feature [2] can also be put to good use by transforming the resulting program into an equivalent form that can be backtracked over, even in parallel, to perform combinatorical search (whereas assertz/retract in Prolog is destructive and cannot be "undone" automatically to search new states). But as you also can see from the plan description text, a natural-language description of the problem that an LLM can comprehend only helps so much since it's highly repetitive in stating hundreds of facts about eg. current and desired package locations. This is where Prolog comes in since you can state those ground facts in clause format, or a DSL format, or a database (as in realistic applications where Prolog facts are generated or Prolog is deployed alongside a backend for integration with other services for eg. messaging in a a logistics app). Still, the combination of an LLM (excelling at language tasks) and Prolog (for actually searching a combinatorical space spanned by clauses extracted in a natural way), just as a human reach out to a pocket calculator for certain tasks, sure is powerful and has been also explored recently in one form or another in academia eg [3] also referenced elsewhere in this thread. And this isn't even using tool calling and iterative re-prompting, etc. [1]: https://quantumprolog.sgml.net/container-planning-demo/part1... [2]: https://quantumprolog.sgml.net/container-planning-demo/part2... [3]: "LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworks" at <https://arxiv.org/abs/2402.01817> PROMPT (submitted as-is, including typos): Write a Prolog program to solve the following logistics problem: Packages must be transported from and to locations inside the six cities named city1, city2, city3, city4, city5, and city6) using trucks, and between those citiesusing airplanes. Each city has two locations: city1 has locations city11 and city12, city2 has locations city21 and city22, city3 has locations city31 and city32, city4 has locations city41 and city42, city5 has locations city51 and city52, andcity6 has locations city61 and city62. Initially, package1 is located at city21, package2 is located at city12, package3 is located in city11, package4 is located in city11, package5 is located in city42, and package6 is located in city31. The aim is to move package1 to city21, package2 to city62, package3 to city61, package4 to city32, package5 to city62, and package6 to city12. In each city, at the locations city11, city21, city31, city41, city51, and city61, there's a truck to transport packages to other locations within the city. There are moreover two airplanes, named plane1 and plane2, located at the airport of city1 at location11 and at the airport of city2 at location22. The airports of the other cities have the locations location32 in city3, location42 in city4, location52 in city5, and location62 in city6. To move packages, a sequence of actions as follows must be performed until the packages have reached their destination. The action load_truck will place a package in a truck at a given location. The action unload_truck will unload a package from a truck at a given location. The action fly_airplane will fly an airplane from one airport to another. The action unload_airplane will unload a package from an airplane at a given location. The action load_airplane will load an airplane with a package at a given location. RESPONSE: % State facts % Initial state of the packages package(package1, city21). package(package2, city12). package(package3, city11). package(package4, city11). package(package5, city42). package(package6, city31). % Initial state of the trucks truck(city11, city11). truck(city21, city21). truck(city31, city31). truck(city41, city41). truck(city51, city51). truck(city61, city61). % Initial state of the airplanes airplane(plane1, city11). airplane(plane2, city22). % Action clauses % load_truck :- load a package into a truck at a given location load_truck :- % If the given location is valid for loading a truck location(Loc, City), package(Pkg, Loc), truck(T, Loc), % Remove the package from its current location retract(package(Pkg, Loc)), % Add the package to the truck assertz(package(Pkg, T)), % Assert the new state assertz(truck(T, Loc)). % unload_truck :- unload a package from a truck at a given location unload_truck :- % If the given location is valid for unloading a truck location(Loc, City), package(Pkg, T), truck(T, Loc), % Remove the package from the truck retract(package(Pkg, T)), % Add the package to its destination location assertz(package(Pkg, Loc)), % Assert the new state assertz(truck(T, Loc)). % fly_airplane :- fly an airplane from one airport to another fly_airplane :- % If the airplane is at a valid airport airplane(plane, Loc), % If there is a valid destination airport airport(Dest, DestCity), % Remove the airplane from its current location retract(airplane(plane, Loc)), % Add the airplane to the destination airport assertz(airplane(plane, Dest)). % unload_airplane :- unload a package from an airplane at a given location unload_airplane :- % If the given location is valid for unloading an airplane location(Loc, City), package(Pkg, A), airplane(plane, A), % Remove the package from the airplane retract(package(Pkg, A)), % Add the package to its destination location assertz(package(Pkg, Loc)), % Assert the new state assertz(airplane(plane, Loc)). % load_airplane :- load a package into an airplane at a given location load_airplane :- % If the given location is valid for loading an airplane location(Loc, City), package(Pkg, Loc), airplane(plane, A), % Remove the package from its current location retract(package(Pkg, Loc)), % Add the package to the airplane assertz(package(Pkg, A)), % Assert the new state assertz(airplane(plane, Loc)). % Target state target :- package(package1, city22), package(package2, city62), package(package3, city61), package(package4, city32), package(package5, city62), package(package6, city12). % Main predicate to solve the problem solve :- ... % BOGUS MAIN AND HELPER CLAUSES OMITTED