Forums: » Register « |  Free Tools |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support |
 User Name: Password: Remember me

New Free Tools on Dev Shed!
We're Excited to announce that Dev Shed now has 70 free tools on the site. To learn more, click here!

 Add This Thread To: Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb
 « Previous Thread | Next Thread » Thread Tools Search this Thread Rate Thread Display Modes
 Dev Shed Forums Sponsor:
#1
November 9th, 2012, 08:48 AM
 KING-OLE
Registered User

Join Date: Nov 2012
Location: Texas
Posts: 24
Time spent in forums: 5 h 54 m 45 sec
Reputation Power: 0
String to Int conversion speed

Using Python 2.6, I'm trying to solve a math problem, I need to convert a string with 10^5 (100,000) numeric characters to an integer.

Using n=int(s) takes it 1 minute and 45 seconds.

I can live with that, but I need to do the same for 10^6, 10^7, etc. until I find the answer I'm looking for, and the time it takes for the int(s) conversion is getting very long.

Is there a faster string-to-int conversion that anyone knows off?

Thanks,

Ole

#2
November 9th, 2012, 09:55 AM
 b49P23TIvg
Contributing User

Join Date: Aug 2011
Posts: 4,169
Time spent in forums: 1 Month 3 Weeks 2 Days 10 h 24 m 44 sec
Reputation Power: 455
Your problem looks similar to another recent post which had base 10 integers of 60 thousand digits. For that particular problem there was no reason to convert the string to an integer. Please describe, at a higher level, the problem you're trying to solve so we can consider other approaches.
__________________
[code]Code tags[/code] are essential for python code!

#3
November 9th, 2012, 10:11 AM
 KING-OLE
Registered User

Join Date: Nov 2012
Location: Texas
Posts: 24
Time spent in forums: 5 h 54 m 45 sec
Reputation Power: 0
Quote:
 Originally Posted by b49P23TIvg Your problem looks similar to another recent post which had base 10 integers of 60 thousand digits. For that particular problem there was no reason to convert the string to an integer. Please describe, at a higher level, the problem you're trying to solve so we can consider other approaches.

For this problem there is a reason to convert it to an integer, as I need to count how many prime factors below 100,000 it contains.

As an example, an integer consisting of 10^1 (10) 1's, have the following prime factors: (11, 41, 271 & 9091).

An integer consisting of 10^2 (100) 1's, have the following prime factors (not checking the previous found): (101, 251, 3541, 5051, 21401, 25601, 27961 & 60101).

At one point (10^n 1's) there will be no more prime factors below 100K. I'm trying to find the ones left over.

#4
November 9th, 2012, 11:38 AM
 b49P23TIvg
Contributing User

Join Date: Aug 2011
Posts: 4,169
Time spent in forums: 1 Month 3 Weeks 2 Days 10 h 24 m 44 sec
Reputation Power: 455
zowie!

j (executable Iverson notation) converts a random string of 1e5 digits to an integer in a flash---well under a second. I asked for the prime factors which sent my computer into a page file hell which j was clever enough to abort. I expect that had I merely asked j to check for a 0 remainder on divisions by the primes less than 100000 there'd have been no trouble. OK, there's a problem with old python (or with your code).

Upgrade to a new python 2.7 or to python3.
The conversion takes place in a blink.
Code:
```\$ python3
Python 3.2.3 (default, Oct 19 2012, 19:53:16)
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import random,string
>>> s=''.join(string.digits[random.randrange(10)]for i in range(100000))
>>> i=int(s)
>>> ```

Alternatively, you could try the problem with the gnu multi-precision library, gmp.

Oh! You could investigate theorems about repdigits.

Last edited by b49P23TIvg : November 9th, 2012 at 11:42 AM.

#5
November 9th, 2012, 11:50 AM
 KING-OLE
Registered User

Join Date: Nov 2012
Location: Texas
Posts: 24
Time spent in forums: 5 h 54 m 45 sec
Reputation Power: 0
With all that being said, how long did the i=int(s) take on your computer with Python 3.2.3?

As far as I recall, I did some speed tests between different Python versions, and I compared 2.6, 2.7 and 3.something. The 2.6 was the fastest (not sure why), which is why I'm still using it.

Thanks.

#6
November 9th, 2012, 12:43 PM
 b49P23TIvg
Contributing User

Join Date: Aug 2011
Posts: 4,169
Time spent in forums: 1 Month 3 Weeks 2 Days 10 h 24 m 44 sec
Reputation Power: 455
Takes 13 seconds for 1 conversion of a million digits.

p.py:

for i in range(1):
a = int('1'*(int(1e6)))

\$ time python3 p.py

real 0m12.385s
user 0m12.345s
sys 0m0.008s

j converts a character array of one million 1's to an extended precision integer in 0.03 seconds. Trust me about the notation or, better yet, learn j.
Code:
```   ts'".''x'',~1e6#''1'''  NB. ts reports time and space
0.026998 3.04111e7```

Here's a solution in j. j is now open source but wasn't. I think we haven't yet replaced custom arbitrary precision integer code with gmp. gmp is faster. (libgmp is also a huge library. That, and perhaps license problems have prevented python from incorporating it.) The run time for all the divisions gets fairly long. And the space used to solve the problem "all at once" is too big. I've got 4Gb RAM on an INTEL new-nomenclature-2 processor running between 2 and 3 GHz Lenovo T500 laptop computer.
Code:
```   p=: i.&.(p:^:_1)1e5  NB. primes less than 1e5

(4&{.,_4&{.)p        NB. first 4 and last 4, for your happy verification
2 3 5 7 99961 99971 99989 99991

]repunit=:".'x',~1e3#'1'   NB. repunit has 1000 ones.
1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111  1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111  11111111111111111111111111111111111111111111111111111111...

mask=: 1(0=|&repunit)\p

(,mask)#p
11 41 73 101 137 251 271 401 751 1201 1601 3541 4001 5051 9091 21001 21401 24001 25601 27961 60101 76001
```

Last edited by b49P23TIvg : November 9th, 2012 at 12:46 PM.

#7
November 9th, 2012, 01:03 PM
 Dietrich
Contributing User

Join Date: Feb 2005
Posts: 575
Time spent in forums: 5 Days 4 h 14 m
Reputation Power: 63
I simply used module timeit.
Function int(s) of the 100000 digit string used by b49P23TIvg takes about
1.2 seconds with Python273 and Python323
0.4 seconds with Python330
__________________
Real Programmers always confuse Christmas and Halloween because Oct31 == Dec25

Last edited by Dietrich : November 9th, 2012 at 01:41 PM.

#8
November 9th, 2012, 01:29 PM
 KING-OLE
Registered User

Join Date: Nov 2012
Location: Texas
Posts: 24
Time spent in forums: 5 h 54 m 45 sec
Reputation Power: 0
Thanks guys.

Before I dive into J, which sounds very interesting, I'd like to "crack the nut" with Python.

I do not understand how you can do it that fast in Python 2.7 and above.

Could I ask you a favor and try this code for me?

Code:
```from datetime import datetime

def timenow():
ts=str(datetime.now())
return ts[11:22]

print "Before creating string   : ",timenow()

s1=""
s2=s1.ljust(10**6,"1")

print "Before converting to int : ",timenow()

n1=int(s2)

print "Int converted            : ",timenow()```

The output from this is:

Code:
```c:\Python26>python p133st.py
Before creating string     :  12:25:26.74
Before converting to int   :  12:25:26.74
Int converted              :  12:27:13.68

c:\Python26>```

As you can see, it takes about 1:45 minutes to convert a 1,000.000 digit string to an integer on my 2.6.

This is a pretty new i7 I have.

Thanks,

Ole

#9
November 9th, 2012, 01:47 PM
 b49P23TIvg
Contributing User

Join Date: Aug 2011
Posts: 4,169
Time spent in forums: 1 Month 3 Weeks 2 Days 10 h 24 m 44 sec
Reputation Power: 455
Output: (11 seconds)
Code:
```\$ python p.py
Before creating string   :  13:44:22.05
Before converting to int :  13:44:22.05
Int converted            :  13:44:32.75
\$```

Slight change to the program:
Code:
```import datetime

now = datetime.datetime(2012,11,9).now#year, month, day[, hour[, minute[, second[, microsecond[,tzinfo]]]]])

def timenow():
ts=str(now())
return ts[11:22]

print "Before creating string   : ",timenow()

s1=""
s2=s1.ljust(10**6,"1")

print "Before converting to int : ",timenow()

n1=int(s2)

print "Int converted            : ",timenow()```

Last edited by b49P23TIvg : November 9th, 2012 at 01:50 PM.

#10
November 9th, 2012, 01:52 PM
 KING-OLE
Registered User

Join Date: Nov 2012
Location: Texas
Posts: 24
Time spent in forums: 5 h 54 m 45 sec
Reputation Power: 0
Wow, that's quite an improvement in speed - roughly 10 times faster.

What version of Python did you run this on?

Thanks again for your help.

#11
November 9th, 2012, 01:59 PM
 b49P23TIvg
Contributing User

Join Date: Aug 2011
Posts: 4,169
Time spent in forums: 1 Month 3 Weeks 2 Days 10 h 24 m 44 sec
Reputation Power: 455
\$ python
Python 2.7.3 (default, Sep 26 2012, 21:51:14)
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

#12
November 9th, 2012, 02:04 PM
 KING-OLE
Registered User

Join Date: Nov 2012
Location: Texas
Posts: 24
Time spent in forums: 5 h 54 m 45 sec
Reputation Power: 0
Thanks - I'll try with 2.7.3.

Sorry I can't rep you right now, but I am so new that I don't have those rights yet.

I'll come back and rep you in a month.

#13
November 9th, 2012, 02:27 PM
 KING-OLE
Registered User

Join Date: Nov 2012
Location: Texas
Posts: 24
Time spent in forums: 5 h 54 m 45 sec
Reputation Power: 0
Hmm, it looks pretty much the same on my computer for both versions. Actually 1 second faster on 2.6...

Code:
```
c:\Python27>python p133st.py
Before creating string   :  13:21:07.63
Before converting to int :  13:21:07.63
Int converted            :  13:22:54.37

c:\Python27>cd\python26

c:\Python26>python p133st.py
Before creating string   :  13:23:12.38
Before converting to int :  13:23:12.38
Int converted            :  13:24:58.05

c:\Python26>
```

I do have other things running on it, but it's an i7 with 8 cores, so ...

#14
November 9th, 2012, 02:53 PM
 Dietrich
Contributing User

Join Date: Feb 2005
Posts: 575
Time spent in forums: 5 Days 4 h 14 m
Reputation Power: 63
Done on a CORE i5 machine:
Code:
```'''mpm_timing2.py
time a function with mpmath.timing()
time int(s) where s is a string of 1 million digits

downloaded Windows installer file (for Python27 to Python33)
mpmath-0.17.win32.exe
from
http://code.google.com/p/mpmath/

mpmath.timing(f, *args, **kwargs)
Returns time elapsed for evaluating f() in seconds
arguments can be passed to time the execution of f(*args, **kwargs)

compares well with the Python module timeit
'''

import mpmath as mpm
import sys

print("Python version:\n %s\n" % sys.version)

# create a numeric test string of one million sevens
s = '7'*1000000

# now time Python function int(s)
print("int(s) = %f seconds" % mpm.timing(int, s))

'''my result >>>
Python version:
2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)]
int(s) = 117.061921 seconds

Python version:
3.2.3 (default, Apr 11 2012, 07:15:24) [MSC v.1500 32 bit (Intel)]
int(s) = 113.484790 seconds

Python version:
3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:55:48) [MSC v.1600 32 bit (Intel)]
int(s) = 31.143113 seconds
'''```

You can see that Python as an interpreted language just comes out slow for your purposes. You might want to check out compiled languages like C or C++ for executable speed.

#15
November 9th, 2012, 02:56 PM
 KING-OLE
Registered User

Join Date: Nov 2012
Location: Texas
Posts: 24
Time spent in forums: 5 h 54 m 45 sec
Reputation Power: 0
Thanks for your testing.

FYI, I just added 3.3.0, and corrected the print syntax, and it now looks MUCH better.

Code:
```c:\Python33>python p133st.py
Before creating string   :  13:54:22.43
Before converting to int :  13:54:22.43
Int converted            :  13:54:32.21```

EDIT: Oh, and C/C++ does not hold integers that large.

 Viewing: Dev Shed Forums > Programming Languages > Python Programming > String to Int conversion speed

## Developer Shed Advertisers and Affiliates

 Thread Tools Search this Thread Search this Thread: Advanced Search Display Modes Rate This Thread Linear Mode Rate This Thread: 5 : Excellent 4 : Good 3 : Average 2 : Bad 1 : Terrible

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts vB code is On Smilies are On [IMG] code is On HTML code is Off
 View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox Forum Jump Please select one User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home -------------------- Programming Languages    PHP Development        PHP FAQs and Stickies    Perl Programming        Perl FAQs and Stickies    C Programming        C Programming FAQs and Stickies    Java Help        Java FAQs    Python Programming        Python Programming FAQs    Ruby Programming        Ruby Programming FAQs    Game Development        Game Development FAQs Programming Languages - More    ASP Programming        ASP Programming FAQs    .Net Development        .Net Development FAQs    Visual Basic Programming        Visual Basic Programming FAQs    Software Design        Software Design FAQs    ColdFusion Development        ColdFusion Development FAQs    Delphi Programming        Delphi Programming FAQs    Regex Programming        Regex Programming FAQs    XML Programming        XML Programming FAQs    Other Programming Languages        Other Programming Languages FAQs Web Design    HTML Programming        HTML Programming FAQs    JavaScript Development        JavaScript Development FAQs    CSS Help        CSS Help FAQs    Flash Help        Flash Help FAQs    Photoshop Help        Photoshop Help FAQs    Web Design Help        Web Design Help FAQs    Website Critiques        Website Critiques FAQs    Search Engine Optimization        Search Engine Optimization FAQs Mobile Programming    Mobile Programming        Mobile Programming FAQs    iPhone SDK Development        iPhone SDK Development FAQs    Android Development        Android Development FAQs    BlackBerry Development        BlackBerry Development FAQs Web Site Management    Business Help        Business Help FAQs    Development Software        Development Software FAQs    Scripts        Scripts FAQs Databases    Database Management        Database Management FAQs    DB2 Development        DB2 Development FAQs    MySQL Help        MySQL Help FAQs    PostgreSQL Help        PostgreSQL Help FAQs    Firebird SQL Development        Firebird SQL Development FAQs    MS SQL Development        MS SQL Development FAQs    Oracle Development        Oracle Development FAQs    LDAP Programming        LDAP Programming FAQs System Administration    Mail Server Help        Mail Server Help FAQs    Apache Development        Apache Development FAQs    Security and Cryptography        Security and Cryptography FAQs    Antivirus Protection        Antivirus Protection FAQs    DNS        DNS FAQs    IIS        IIS FAQs    Networking Help        Networking Help FAQs    FTP Help        FTP Help FAQs Operating Systems    BSD Help        BSD Help FAQs    Linux Help        Linux Help FAQs    UNIX Help        UNIX Help FAQs    Windows Help        Windows Help FAQs    Mac Help        Mac Help FAQs Web Hosting    Web Hosting        Web Hosting FAQs    Free Web Hosting        Free Web Hosting FAQs    Web Hosting Requests        Web Hosting Requests FAQs    Web Hosting Offers        Web Hosting Offers FAQs Computer Hardware    Computer Hardware    CPUs        CPUs FAQs    Cooling        Cooling FAQs    Embedded Programming        Embedded Programming FAQs    Motherboards        Motherboards FAQs    Multimedia Hardware        Multimedia Hardware FAQs Other    Dev Shed Lounge        Dev Shed Lounge FAQs    Development Articles        Development Articles FAQs    Beginner Programming        Beginner Programming FAQs    Hire A Programmer        Hire A Programmer FAQs    Project Help Wanted        Project Help Wanted FAQs Latest News Updated Hourly    Technology News    Business News    Science News Forum Information    Forum Rules/Guidelines        Forum Rules/Guidelines FAQs    Forum Announcements        Forum Announcements FAQs    Dev Shed Gaming Center        Go to the Dev Shed Battle Arena        Go to the Dev Shed Arcade Games        Go to the Legend of the Green Dragon    Suggestions & Feedback        Suggestions & Feedback FAQs

 Forums: » Register « |  Free Tools |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support |

Powered by: vBulletin Version 3.0.5
Copyright ©2000 - 2013, Jelsoft Enterprises Ltd.

© 2003-2013 by Developer Shed. All rights reserved. DS Cluster - Follow our Sitemap