#1
  1. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jul 2013
    Location
    Malta
    Posts
    2
    Rep Power
    0

    Which skills should I focus on?


    Hello There,

    It's nice to see Devshed still going strong, I used to use this site back in the 90's but havent had to write much code since then.

    Right then, I will shortly have some more time on my hands to pursue a startup idea I've had for about 18 months. I used to code for a living (COBOL in the 1980's and PHP in the 1990s) though I have not even looked at a line of code since about 2003. I am effectively a newbie again and ready to re-learn everything, even the binary language of moisture vaporators if necessary.

    I am leaning towards using Python for a variety of reasons but also use of the scrapy web-scraping/mining library.

    My idea is this....

    I need to launch server-client internet-gambling games automatically (I will take the website url and location of the game launch links from a separate DB that a crawler will build)

    when the game starts to download to the users machine I wish to analyse the data packets to identify the specific game. Many games have similar names, only by analysing the game itself can I determine what game it is. These games are usually built by 3rd party designers. My idea is to map which designers products are in use on which gambling sites, save that info and build up trend based product distribution info.

    I then generate meta-data to identify that game, the company that designed it (from another db) the website it is featured on and other criteria and save that in a DB to populate the reporting of my product.

    I am looking at deep packet inspection software to help me analyse the data packets of each game however that seems to be over-kill.

    Is python a good choice for such a task or should I be looking at another technology to help me identify these games.

    I hope I explained the question clearly.

    Thank you in advance for your time.

    Adam
  2. #2
  3. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    May 2013
    Location
    Usually Japan when not on contract
    Posts
    240
    Rep Power
    12
    Sounds like you might be headed in dubious directions; if not now then after some new things occur to you...

    That said, there are (many) libraries in Python to handle any sort of packet or bit-level inspection/manipulation you require.

    If you need lightning-fast reactions, like to intercept/respond within a latency window the way that the (slowly self-digesting) firms that game the latency between financial exchange systems do, then you'd want to get as close to the hardware as possible. That means very clean C, which is hard, so folks with those sort of needs write Haskell programs that write their C programs -- the coding of which is slightly less hard and much more flexible.
  4. #3
  5. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jul 2013
    Location
    Malta
    Posts
    2
    Rep Power
    0
    Thanks for the insight it is greatly appreciated.

    No I do not need to respond within a specific latency window although like everything responses would ideally be quicker rather than slower.

    I'm glad to hear Python has such libraries. My main fear is that I take a few months to get up to a somewhat competent level with the language and it turns out to not be capable of what I need it for.

    Is the 'dubious' discussion related to the fact this is essentially a web-scraping / data-aggregation idea?
  6. #4
  7. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    May 2013
    Location
    Usually Japan when not on contract
    Posts
    240
    Rep Power
    12
    re: dubiosity
    Its not so much that you're scraping as what you're scraping. You'll no doubt stumble on a few ways to get over on someone here or there...

    But back to Python... its easy to learn, is surprisingly free of baggage and sports an overwhelmingly huge host of utility packages. If you find you need something "deeper" (which really boils down to "faster") then you'll be happy to know that you can write modules in C and use them directly within Python as native Python modules (and this isn't even very hard to do).

IMN logo majestic logo threadwatch logo seochat tools logo