The main insurmountable difficulty with Racket is garbage collections. Eventually (actually pretty frequently) there will be a major garbage collection (as opposed to a minor GC, in the context of a generational GC), which tends to take more than 100 milliseconds even with a bare minimum of objects allocated. That is a game-breaker for this sort of... game.