Show HN: Jev Plays Pokémon Red

hckrnws

Show HN: Jev Plays Pokémon Red

(jev-pokemon.vercel.app)

183

16h

by pancomplex

Hey HN! Wanted to share a fun project I've been hacking on. Given Jev can make decisions really fast (but not fast enough to play Doom yet sadly), I wanted to try and push it to play a more complex game than Tetris. So I went with Pokémon.

I've spent endless hours playing this game as a child so building this was a ton of fun.

I open sourced everything in case you want to hack on it yourself here: https://github.com/christianmat/jev-pokemon

The game is being streamed live including the tokens and cost - hopefully we get all the badges and don't get stuck in a cave :)

stusmall
3510h
This is so interesting to watch. For a couple minutes I was in awe of how quick and cheap it was. Then I saw just how bad the decision are and how it would get stuck in strange loops of going in and out of the same door to no end.
This seems like a technology heading in the right direction but not quiet there yet. Excited for what they are cooking up but probably won't start building around it yet.
1. binlog
  2510h
  This entire conversation around Jev seems weird to me. Like... we started from neural nets that could do basic decision making and classifications pretty well, then trained larger and larger language models to get to where we are now. Now suddenly everyone is going crazy because someone trained a smaller model that is adequate at making decisions? We already went through the "look this AI can play pokemon terribly" phase like a decade ago.
  1. c7b
    59h
    A pre-trained universal classifier that can replace specifically-trained ones would have been considered just as much science fiction in the 2010's as the capabilities of modern LLMs. I'm not sure Jev is actually there yet, but at least it sounds theoretically doable today.
    That being said, one thing having been unrealistic 10 years ago and just about possible today doesn't mean that it's going to change the world the same way another technically related, previously-impossible thing did. The Jev hype gives me a bit of the "you're still early to crypto" vibes of some later altcoins. I really like the idea, I think it's going to open up possibilities for using classifiers where we wouldn't or couldn't have trained one before. I'm crossing my fingers for an open weights version to drop. But it's still just a classifier, people have built similar things before Jev, the one thing that really stands out about it is their ability to generate hype.
    someothherguyy
    4
2. jbjbjbjb
  18h
  Jev is for single shot classification, not multi-step RL environments with delayed reward and explore/exploit. My guess is it would go through the door with high confidence every time unless you change the input to add the history.
  1. pancomplex • OP
    08h
    This runs entirely on Jev as the only AI with a typescript harness that feeds it selective context.
3. ralusek
  410h
  The exact message I sent my friend this morning:
  > the most interesting thing about this jev stuff
  > is that people are seemingly like
  > completely disinterested in how smart it actually is
  > I haven't even heard it mentioned a single time how it actually compares to other LLMs coming up with their own classifications. Just: it's fast and cheap
  After watching a few minutes of this it makes me think that maybe we should be a little more interested in how smart it is.
  1. johnsmith1840
    07h
4. pancomplex • OP
  110h
  Like others have mentioned in this post, I think a mix of models like Jev for simple stuff + a smarter reasoning model for more strategic thinking is the optimal solution. This experiment however is purely Jev. Which sometimes can be kinda dumb.
  1. JamesSwift
    03h
    Well in a lot of ways this demo actually is a mix of jev for simple stuff + a very intelligent harness to fill in the rest. Pay attention to the "jev counter" on when api calls actually happen and the kinds of decisions its making. Its very rarely in a tight loop, and when it is (eg in an item menu) it tends to randomly walk through options.
    Its very cool, but the harness is doing a _ton_ of heavy lifting.
ac2u
07h
Cool project, comes with a little too much guidance in the harness though IMO (pathfinding, textual milestones etc). (The author is very upfront about this in their README though)
I think if it was combined with a regular vLLM it could be really interesting, especially watching the reasoning logs.
Bonus points if it was one of the latest open models that somehow had all prior training knowledge of Pokemon abliterated so it was reasoning as an intelligent persona that had no knowledge of even the concept of Pokemon.
MitPitt
311h
This is kinda chill to have in the background. I wish there were livestreams showing live reasoning of top models which are currently trying to solve cancer or whatever. Imagine the pogs in chat when it does.
1. pancomplex • OP
  011h
  People need to be live streaming their AI more!
testaccount28
211h
with such a fat harness, this is more like watching a walk thru play the game.
1. laszlokorte
  110h
  Yeah I would have expected it to only decide which button to press, not something abstract like the choice of "go east to lavender town" for the goal of "in lavender town, climb the pokemon tower"
dang
27h
Dare we have two Pokemon-playing-AI threads at the same time?
Teaching a World Model to Play Pokemon - https://news.ycombinator.com/item?id=49849907
1. fylo
  03h
  Ai pokemon red speed run chart seems like a cool benchmark
ViscountPenguin
16h
The choices Jev has here feel very railroady, it seems like something very significantly dumber could beat the game with these options.
1. pancomplex • OP
  05h
  I challenge you to fork the repo and solve it faster using less tokens/cost than Jev :)
rickintoplace
616h
That's actually fun to watch. Did you experiment with nicknaming before you turned it off? I'd be a little curious to see how it behaves.
1. pancomplex • OP
  516h
  Jev can't come up with original text, but I did consider giving it a list of hilarious names.
[deleted]
06h
[deleted]
lwarfield
411h
Looking at the diagram in the gh repo, it looks like this is entirely jev. Are there any examples of people having a big model like Fable handle high level goals?
1. pancomplex • OP
  010h
  Frontier reasoning models do pretty well in Pokemon: https://github.com/benchflow-ai/pokemon-gym
  The interesting thing here imo is the cost and latency. So far we're at 4 badges for less than $0.5
stusmall
110h
>but not fast enough to play Doom yet sadly
Did I miss something? I thought one of the demo videos was it doing pretty decent at the first level of Doom?
1. pancomplex • OP
  010h
  In my experience it was too slow to do an FPS with 30 ticks per second reliably. It gets killed too fast.
flockonus
06h
The decisions are quick, but look good as random w/ tons of back and forth. Are you at least feeding back some of its previous decisions on to state?
djhworld
110h
What's not clear to me on the video is whether jev is doing the button presses for controlling the character to move around.
The "Jev calls" counter only seems to increment at junction points like battles, conversation prompts, menus etc.
Is something else moving the character around?
1. pancomplex • OP
  010h
  It's connected to the ROM of the actual game, so it can see a lot of things. It has multiple tools available, including being able to move to coordinates.
  All in the OSS repo if you wanna play around with it: https://github.com/christianmat/jev-pokemon
hummusFiend
18h
Great work!
Super cool to see it do the whole game. I spent my fable budget building something similar this week but only drove it to Brock. I like the "current focus" framing too.
1. pancomplex • OP
  08h
  Thank you! And interesting data point on Fable burning out on tokens so early.
dochaus
06h
He confirms what I long suspected, all you need is to cheese Charizard and flamethrower
rubicon33
05h
Pretty awesome! Just curious are you vibe coding this?
[deleted]
011h
[deleted]
dmitrygr
110h
Considering it just made Charizard forget its only fire-type move "Ember" to learn "Counter", I note no signs of intelligence.
1. pancomplex • OP
  010h
  Rookie mistake clearly..
361994752
211h
watched it stuck at rocket hideout for 10 mins.... let me check 1hr later to see if it can find a way out
1. jumploops
  010h
  It’s currently stuck at an elevator and deciding to teach Pokemon various TMs and HMs instead of progressing… pretty hilarious!
sync1117
03h
i need to come up with something more creative and interesting than video games as benchmarks
theturtletalks
210h
Is Frigade going to use Jev to do object detection?
1. pancomplex • OP
  110h
  Working on it :)
zaik
09h
Seems like it got stuck on a Ghost enemy.
avaer
210h
I wish jev took in images so we could do this generically for any game, without memhacks. I'm sure that's coming.
You could front this with an image -> text model but that would be much lower quality vs latency, and the whole point of doing it with a decision model is remove the latency.
Games are a really interesting testing ground for robotics; if we can solve game playing (incl 3d) we could embody "system one" intelligence into robots that have something emulating general reflexes without needing to fine tune.
1. pancomplex • OP
  19h