Interacting with another program

I am trying to write a python program to interact with a separate program (UCI chess engine). To do so I need to read and write via stdin and stdout. Thus far I have had limited success and thought I would reach out for some help.

I have been trying to use the SUBPROCESS module and have been able to capture some limited output using the CHECK_OUTPUT command. The main issue with this is that it closes the instance of the program I am interacting with as soon as the output is returned. Ideally I would like to keep the program running in the background rather than opening and closing many times.

At any rate, I seem to be making this harder than it should be. I think it should be a fairly simple task to do but it is not turning out that way.

To summarize, I am trying to:
1. Open chessengine.exe
2. Send command string
3. Read output
4. Return to step 2, etc.

Thanks in advance for any suggestions.

time to sleep and I haven't gotten my adorable example to work.

You can get assistance at freechess.org
or read the instructions for connecting a chess engine to xboard

good night.

I tried a few different values of bufsize. Sorry.
I'll report the __exit__ undefined in python2.7 error as python bug.

Thank you very much for the suggestion. The universal_newlines option was a big help. I was unaware of it and was trying to encode / decode the output so that makes things a bit cleaner.

It is quite close, but the p.stdout.readline() does not capture all of the text. For example, sending the command UCI to the engine receives several lines of text in return but I am only capturing the first line. I tried using p.stdout.read() and p.stdout.readlines() and they both hang.

I also tried using a loop to iterate through the lines but that is hanging as well. I tried:

Any other suggestions would be greatly appreciated.

Your code will print alternate lines from the input as does this example. python3 simplifies the demonstration, I don't need an external file.




You might have meant

Thanks for the comments but I am still having issues retrieving all of the slave program's output. Commands have been highlighted for reference and the output of the slave program is in the normal text.

Here is an example command line session I am trying to duplicate.

Stockfish 2.1.1 JA by Tord Romstad, Marco Costalba and Joona Kiiski
uci
id name Stockfish 2.1.1 JA
id author Tord Romstad, Marco Costalba and Joona Kiiski

option name Use Search Log type check default false
option name Search Log Filename type string default SearchLog.txt
option name Book File type string default book.bin
option name Best Book Move type check default false
option name Mobility (Middle Game) type spin default 100 min 0 max 200
option name Mobility (Endgame) type spin default 100 min 0 max 200
option name Passed Pawns (Middle Game) type spin default 100 min 0 max 200
option name Passed Pawns (Endgame) type spin default 100 min 0 max 200
option name Space type spin default 100 min 0 max 200
option name Aggressiveness type spin default 100 min 0 max 200
option name Cowardice type spin default 100 min 0 max 200
option name Minimum Split Depth type spin default 4 min 4 max 7
option name Maximum Number of Threads per Split Point type spin default 5 min 4
max 8
option name Threads type spin default 2 min 1 max 32
option name Use Sleeping Threads type check default false
option name Hash type spin default 32 min 4 max 8192
option name Clear Hash type button
option name Ponder type check default true
option name OwnBook type check default true
option name MultiPV type spin default 1 min 1 max 500
option name Skill Level type spin default 20 min 0 max 20
option name Emergency Move Horizon type spin default 40 min 0 max 50
option name Emergency Base Time type spin default 200 min 0 max 30000
option name Emergency Move Time type spin default 70 min 0 max 5000
option name Minimum Thinking Time type spin default 20 min 0 max 5000
option name UCI_Chess960 type check default false
option name UCI_AnalyseMode type check default false
uciok
isready
readyok
position fen 8/4K3/4NN2/p3p3/rnp1p3/1pk5/bp1n4/qrb1N3 w
go
info depth 1
info depth 1 seldepth 1 multipv 1 score cp -4129 nodes 106 nps 1139 time 93 pv e
7d6
info nodes 106 nps 1139 time 93
bestmove e7d6

Here is the program that I am trying, which is based on your first example.



and here is the program output.

>>>
Stockfish 2.1.1 JA by Tord Romstad, Marco Costalba and Joona Kiiski

id name Stockfish 2.1.1 JA

id author Tord Romstad, Marco Costalba and Joona Kiiski



>>>

And now the question - why does my program output not match the command line output?

(a decade ago?) I got this sort of thing to work with fluent as the program I wanted to control.

The idea is to identify the interactive prompt returned by the program. Maybe this will work on your system? It doesn't work on mine. I took the isready command out of your command list. The code issues two commands for each command you actually care about: the one you want followed by isready. I presume the output from stockfish following isready is always 'readyok'. The program reads output from stockfish until it finds 'readyok' then you can issue the next command paired with isready.

Good luck.


I suppose my trouble is that I don't understand how buffering works on unix.

This works for me. The other post was bad code---Good idea, Wrong implementation. Thank you for pointing out that I was missing new lines from my tests and struggles!!!

I just wanted to follow up and say that I got this to work as well. Thanks so much for all of your help.

I am trying to do something very similar.

I was able to get this approach to work too, but I find that it doesn't work with some other chess engines. I think it has something to do with buffering but I'm not certain. I get a lot of blank lines with some engines.

I am noticing that Crafty has an annotate feature where you can tell it to annotate an entire PGN input file, and an input feature where you can tell it to read commands from an input file. So this might be the solution for chess.

But for the more general question of how to interact with an external terminal program, I am thinking maybe telnet would work better. I am working in Linux so I can make the assumption that console logins are OK.

My approach absolutely depends on knowing what the next "ready" prompt will be. I've also used regular expressions to match a set of distinguishable possible "ready" prompts.

If chess is your interest, the people at
http://www.freechess.org
FICS are helpful. They host discussion a group devoted to interfacing chess engines to other programs. If you write your own chess program it can play at FICS, if you're nice and agree to the rules. They accept variants on available codes. For instance, stockfish is already (given correct recollection) available to play. They might however, accept stockfish with custom weights, or self-learning weights.

(oh boy! There is a program at FICS that "learns" by collecting a database of moves. This is infeasible because the tree of moves is large. I've just cut the number of variables to as low as 6. Cool---stockfish driven by Nelder-Mead.)

im working on the same project. im making a chess playing robotic arm. so i need to communicate with the chess engine and take the output generated by the engine to my program. i tried the codes provided by u guys but its only printing what's already there in the command window "Stockfish 2.2.2 JA by Tord Romstad, Marco Costalba and Joona Kiiski"
that means im not able to write the command strings to the engine
pls help me

I've attached the description of the interface, although I'd think engine-interface.txt comes with stockfish installation.

The code of my post on
October 19th, 2011 11:05 PM
still works. Upon review, the only change I'd make is to change
print(line)
to
sys.stdout.write(line)
And the code work with both python2 and python3.

I'm using current Ubuntu linux distribution.

Do you have a recent installation of python with a current subprocess module?

im using python 2.7 on windows..... can that be the problem. ???
import subprocess as S
getprompt = 'isready'
done= 'readyok'

commands = ('uci', 'position fen 8/4K3/4NN2/p3p3/rnp1p3/1pk5/bp1n4/qrb1N3 w', 'go',)
proc= S.Popen('c:\stockfish.exe', stdin=S.PIPE, stdout=S.PIPE, bufsize=1, universal_newlines=True)

for i in commands:
proc.stdin.write(i + '\n')
proc.stdin.write(getprompt + '\n')
while True:
line= proc.stdout.readline()
print(line)
if done == line[:len(done)]:
break

this is what i used.....

Beats me. Sure looks like similar code. And you claimed to have started stockfish. I don't do windows but thought that the subprocess module was supposed to be operating system independent. Maybe the line endings differ? Maybe you have CR LF to contend with? I don't think that's the trouble though. I'm quite clueless.