#1
  1. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jan 2017
    Posts
    4
    Rep Power
    0

    pySerial .readline() help


    Hello,

    I am writing a Python v3.6.0 program on a Windows PC that takes an input from the Serial port and then parses the input, and replies on the Serial port based on the input.

    I tied COM1 and COM2 together. My Python script running is connected to COM1 and the Arduino serial monitor connected to COM2. I am using the pySerial library's command ".readline()" to send the serial communication bytes to my Python script.

    According to the pySerial docs. the .readline() command will return a string after it receives a '\n' character. However, when I send a string of text through the Arduino serial port monitor without a newline character at the end of it, the .readline() method returns that string. If I do send a string with a newline character, the newline character does get sent with the string to the rest of my script from the .readline() method.

    Does anyone know why how the .readline() method knows to return the 'line' even though there is no newline character at the end of it?

    Code:
    import serial
    
    ser = serial.Serial(port='COM1', baudrate=9600, timeout=0)  # open serial port 
    print("port opened on " + ser.name)
    
    def waiting():
    while(1):
    parse_serial_port(ser.readline()) #this returns after strings are sent via the Arduino Serial Port monitor even without a newline character
    Documentation for the pySerial library: Short introduction — pySerial 3.2.1 documentation


    Thanks,
    Alex
  2. #2
  3. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    May 2009
    Posts
    667
    Rep Power
    40
    timeout=0 should return any character(s) immediately. Check the serial docs to be sure.
  4. #3
  5. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jan 2017
    Posts
    4
    Rep Power
    0
    Originally Posted by dwblas
    timeout=0 should return any character(s) immediately. Check the serial docs to be sure.
    I think that's true for .read(), but for .readline() command it says that it waits for a '\n' character forever without a timeout specified (the default is timeout=0).
  6. #4
  7. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    May 2009
    Posts
    667
    Rep Power
    40
    You did not say what happened when you tried increasing the timeout's value. We do not know what parse_serial_port() does so there is no way to debug any further.
  8. #5
  9. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jan 2017
    Posts
    4
    Rep Power
    0
    When I increased the timeout, it seemed to add a delay to how fast it responds. When I do send bytes over, all the bytes are received and parsed as expected. When I type multiple characters and then hit 'send' on the Arduino serial monitor the python script parses all the characters as if they were one string. If I send one character at a time it bunches up all the characters that came through in the last 2 second window.

    When I don't send anything the text "Nothing happened" gets printed every ~2 seconds as expected. Here is the code for the parse_serial_port function:

    Code:
    def parse_serial_port(line):
    	outgoing = b'' #set this to null so we can check if something happend
    	if(line == b''):
    		print("Nothing happened")
    		return
    	print("incoming: ", end="")
    	print(line + b'', end="")
    	if(line == b'RF'):
    		outgoing = line + b'0000 0000*\r\n'
    		ser.write(outgoing)
    	if(line == b'MG'):
    		outgoing = line + b'0000 0000*\r\n'
    		ser.write(outgoing)
    	if(line == b'RM'):
    		outgoing = line + b'6 000111 0 0 1' + b'1'
    		# outgoing = b'6 000011 0 0 0 RM0000 0000*\r\n' 6 000011 0 0 0 
    		outgoing += b'\r\n'
    		ser.write(outgoing)
    	if(line == b'N1'):
    		outgoing = b'N1*\r\n'
    		ser.write(outgoing)	
    	if(line == "EXIT"):
    		sys.exit(0)
    	if(outgoing == b''):
    		outgoing = line + b'\r\n'
    		ser.write(outgoing)
    	print ("     outgoing: " + outgoing.decode('utf-8'), end="")
    	return
    When I set timeout=0 (and comment out the "Nothing happened" line) the program works as expected, except that the ser.readline() function returns even with no '\n' character. The docs make it seem like it should block and wait forever for that character, but it seems to respond instantly.
    When I type multiple characters and then hit 'send' on the Arduino serial monitor the python script parses all the characters as if they were one string. If I send one character at a time it only parses one character at a time, even though there was no newline character.
  10. #6
  11. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    May 2009
    Posts
    667
    Rep Power
    40
    When I set timeout=0 (and comment out the "Nothing happened" line) the program works as expected, except that the ser.readline() function returns even with no '\n' character.
    Take a look at the in_waiting() function in the docs https://pyserial.readthedocs.io/en/l...erial_api.html Your program says
    "readline() if there is a newline character __and__ readline() if the timeout is reached" (with timeout set to some time interval).
    You can set the timeout to some very large number (or perhaps None?? check the docs) which will only read when a newline is found. You would then have to issue a read once the loop exits to read anything in the pipe at the end of the program. This would require keying in something to exit, which would require more than you have, a GUI or multiprocessing since you are doing 2 things, 1. reading Arduino, 2. checking for exit character.
    Last edited by dwblas; February 7th, 2017 at 10:49 PM.
  12. #7
  13. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jan 2017
    Posts
    4
    Rep Power
    0
    I think it would make sense if it would return the bytes in the buffer when a newline is reached OR the timeout time has passed (instant response when timeout=0, or block forever if '\n' is never received). However, when I have the timeout set to 0 readline() still returns from the function call with the bytes that I had just sent via the Serial Monitor. This makes sense to me when the timeout is not set to 0, but when the timeout is set to 0, it should wait forever until a newline character is reached, right?

    I could do the separate thread and poll the inWaiting() function, but for some reason it appears to work naturally with timeout=0 and no newline character.
    Here is the output to the command line that I get for sending text to the python script via the Arduino serial monitor:


    From Arduino Serial Monitor (no ending character):
    test - no line ending from Arduino serial monitor, timeout=5s
    Command line output from Python script after a 5s delay:
    incoming: b'test - no line ending from Arduino serial monitor, timeout=5s' outgoing: test - no line ending from Arduino serial monitor, timeout=5s



    From Arduino Serial Monitor (newline character appended by the Serial Monitor software):
    test - newline character coming from Arduino serial monitor, timeout=5s
    Command line output from Python script after a 5s delay: incoming:
    b'test - newline character coming from Arduino serial monitor, timeout=5s\n' outgoing: test - newline character coming from Arduino serial monitor, timeout=5s



    From Arduino Serial Monitor ((no ending character):
    test - no line ending from Arduino serial monitor, timeout=0s
    Command line output from Python (instant response - no delay):
    incoming: b'test - no line ending from Arduino serial monitor, timeout=0s ' outgoing: test - no line ending from Arduino serial monitor, timeout=0s



    From Arduino Serial Monitor (newline character appended by the Serial Monitor software):
    test - newline character coming from Arduino serial monitor, timeout=0s
    Command line output from Python (instant response - no delay):
    incoming: b'test - newline character coming from Arduino serial monitor, timeout=0s\n' outgoing: test - newline character coming from Arduino serial monitor, timeout=0s
  14. #8
  15. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Mar 2018
    Posts
    1
    Rep Power
    0

    0 and None are different


    Originally Posted by AlexSneedMiller
    I think that's true for .read(), but for .readline() command it says that it waits for a '\n' character forever without a timeout specified (the default is timeout=0).
    0 is non-blocking (immediate). The serial port default is None. Whatever the timeout is set to will be used for readline(). Infinite timeout is "None". The timeout=<value> is actually a floating point, so you could use something like 0.1 if you wanted a 100ms timeout. You can also set the timeout after the port is opened via the timeout property.

    With timeout=0 you will get something between 0 characters and a line ending in \n. What you get depends on what is currently in the serial buffer when you call readline().

    A timeout isn't really considered an error by pySerial. To check if a timeout occurred, you need to check the last character in the returned buffer. If it isn't a new-line then a timeout occurred.

    You might also want to look at the pexpect-serial package. That's a pretty easy way to match patterns for parsing. I found I needed to set the timeout on the serial open to 0 when using Python 3.6 and Windows to get the expect timeout to work.

    Regards,
    Mike

IMN logo majestic logo threadwatch logo seochat tools logo