#1
  1. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jul 2013
    Posts
    2
    Rep Power
    0

    Making CFHTTP requests inside a CFTHREAD inside a CFLOOP...


    I've been developing an API for FIFA 13 that will allow me to search for cards and if any of the results match my criteria then it will make a bid for that card.

    At the moment I have an index.cfm file that contains a 'Start' search button and this calls a JS function which makes a XMLHttpRequest and I call another file from my application in the .open method which makes a CFHTTP request to the server, loops round the response, and submits another CFHTTP request if I'm submitting a bid on any of the returned results.

    The issue I seem to be running in to though is that there is a max request per second threshold of 3 which I seem to exceeding. I've put the JS function within a setInterval function and set it to every 5 seconds (which should be more than enough) but from looking in FireBug what seems to be happening is that some of the CFHTTP search requests are queuing and then sending all within the same second which is causing my session to timeout.

    What I've now tried to implement is creating a stand-alone page which does a continous CFLOOP and in each loop does the following:

    - Starts a CFTHREAD sleep for 1 second (to stop more than one CFHTTP request being sent per second)
    - Starts a CFTHREAD run that does the search (via CFHTTP request), loops round the results, checks how long the current listing has left and resets the start position if more than 59 mins else sets my bid amount, does a CFHTTP request to make sure I've got enough coins, then does a CFHTTP request to make my bid.
    - Starts a CFTHREAD join.

    The code is as follows:

    <!--- INCLUDE LOGIN FUNCTIONS --->
    <cfinclude template="connect.cfm" />


    <!--- DEFAULT VARIABLES --->
    <cfset searchNo = 0 />
    <cfset numItems = 16 />
    <cfset doSearch = 1 />
    <!--- SET START POSITION AS -16 SO THAT FIRST LOOP STARTS AT POSITION 0 --->
    <cfset startPosition = -16 />


    <!--- OVERWRITE DEFAULT REQUEST TIMEOUT TIME TO 12 HOURS --->
    <cfsetting requestTimeOut = "43200" />


    <!--- DO LOOP --->
    <cfloop condition = "doSearch EQUALS 1">

    <!--- INCREMENT VARIABLE FOR THREAD NAME AND SEARCH POSITION --->
    <cfset searchNo = searchNo + 1 />
    <cfset startPosition = startPosition + 16 />

    <!--- SLEEP FOR 1 SECOND TO AVOID EXCEEDING RPS THRESHOLD --->
    <cfthread action="sleep" duration="1000" />

    <!--- RUN PLAYER SEARCH FUNCTION --->
    <cfthread action="run" name="playerSearch#searchNo#">

    <!--- CALL SEARCH FUNCTION TO MAKE CFHTTP REQUEST --->
    <cfset getPlayer = Application.cfcs.Search.getPlayer(SESSION.phishingKey,SESSION.session Key,"gold","any","any",0,0,0,0,500,0,0,Variables.startPosition,Variabl es.numItems) />


    <!--- IF RESPONSE FROM SEARCH CFHTTP REQUEST IS SUCCESSFUL AND FILECONTENT HEADER HAS RETURNED JSON DATA --->
    <cfif getPlayer.StatusCode EQ "200 OK" AND IsJSON(getPlayer.FileContent)>

    <!--- DESERIALIZE RETURNED JSON SO CAN LOOP ROUND EACH RESULT --->
    <cfset Variables.searchResults = DeserializeJSON(getPlayer.FileContent) />

    <!--- SET LOOP LENGTH --->
    <cfset Variables.numResults = ArrayLen(Variables.searchResults.auctionInfo) />

    <!--- LOOP ROUND RESPONSE --->
    <cfloop from="1" to="#Variables.numResults#" index="i">

    <cfscript>

    // DETERMINE HOW MANY MINUTES CURRENT ITEM LISTING HAS LEFT
    if (Variables.searchResults.auctionInfo[i].tradeState EQ "active") {
    Variables.secondsLeft = Variables.searchResults.auctionInfo[i].expires;
    Variables.timeNow = Now();
    Variables.listingEnds = DateAdd("s",Variables.secondsLeft,Variables.timeNow);
    Variables.minsLeft = DateDiff("n",Variables.timeNow,Variables.listingEnds);
    }

    // IF ITEM LISTING HAS MORE THAN 60 MINS LEFT THEN RE-SET START POSITION
    if (StructKeyExists(Variables,"minsLeft") AND Variables.minsLeft GT 59) {
    Variables.startPosition = 2000;
    }
    // ELSE IF ITEM HAS LESS THAN 60 MINS REMAINING
    else {

    // DEFAULT BID AMOUNT
    Variables.bidAmount = 0;

    // SET BID AMOUNT
    // IF ITEM HAS BUY NOW PRICE AND IT IS LESS THAN QUICK SELL VALUE THEN BID THAT AMOUNT
    if (Variables.searchResults.auctionInfo[i].buyNowPrice NEQ 0 AND Variables.searchResults.auctionInfo[i].buyNowPrice LT Variables.searchResults.auctionInfo[i].itemData.discardValue) {
    Variables.bidAmount = Variables.searchResults.auctionInfo[i].buyNowPrice;
    }
    // ELSE IF QUICK SELL VALUE IS 228,231,235,238,242,245 OR 249 BID 200
    else if (ListFind("228,231,235,238,242,245,249",Variables.searchResults.aucti onInfo[i].itemData.discardValue) AND Variables.searchResults.auctionInfo[i].startingBid LTE 200 AND Variables.searchResults.auctionInfo[i].currentBid LT 200) {
    Variables.bidAmount = 200;
    }

    // GET MY CURRENT COIN TOTAL
    Variables.getCoinTotal = Application.cfcs.Club.getCreditsTotal(SESSION.phishingKey,SESSION.ses sionKey);
    Variables.curCoinsData = DeserializeJSON(Variables.getCoinTotal.FileContent);
    Variables.curCoins = Variables.curCoinsData.credits;

    // IF CURRENTLY HAVE ENOUGH COINS IN ACCOUNT PLACE BID
    if (StructKeyExists(Variables,"curCoins") AND Variables.bidAmount NEQ 0 AND Variables.bidAmount LT Variables.curCoins) {
    Variables.makeBid = Application.cfcs.Bid.PlaceBid(Variables.searchResults.auctionInfo[i]. tradeID,Variables.bidAmount,SESSION.phishingKey,SESSION.sessionKey);
    Variables.bidsMade = Variables.bidsMade + 1;
    } else {
    Variables.bidAmount = 0;
    }
    }

    </cfscript>
    </cfloop>

    </cfif>

    </cfthread>

    <cfthread action="join" name="playerSearch#searchNo#" />

    </cfloop>


    This doesn't seem to be running as it should. It seems to run for the first while but then whenever I use another page to check what cards I've bidded on it seems to have made a few at the start of the loop but then that's it (and it should as there's always cards that match my criteria).

    Can someone please point me in the right direction of where I might be going wrong?
  2. #2
  3. No Profile Picture
    Moderator

    Join Date
    Jun 2002
    Location
    Raleigh, NC
    Posts
    5,264
    Rep Power
    968
    There's really too much going on in your code for me to go over it thoroughly, but can't you just omit all the thread logic and just do a loop, using a sleep() call every 3 iterations to prevent going over 3 at once?
  4. #3
  5. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jul 2013
    Posts
    2
    Rep Power
    0
    Originally Posted by kiteless
    There's really too much going on in your code for me to go over it thoroughly, but can't you just omit all the thread logic and just do a loop, using a sleep() call every 3 iterations to prevent going over 3 at once?
    I've commented out the thread stuff and at the top set a sleep variable for 1 second but this just seems to call the function every second and not necessarily make the cfhttp request every second - they seem to pend and then more than 3 send within one second so it times out the session.

    Is there a way of checking how many active cfhttp requests there are and only submitting a new one when this is less than 3?
  6. #4
  7. No Profile Picture
    Moderator

    Join Date
    Jun 2002
    Location
    Raleigh, NC
    Posts
    5,264
    Rep Power
    968
    Not that I know of.

    I think the problem may be that what you have now first does a one-second sleep on the CURRENT THREAD, but then goes and fires off a SEPARATE THREAD to run the CFHTTP calls. So that second thread (where all your logic is happening) is going on outside of where the sleep is happening.

    Basically I'm saying just try something much simpler, like this (in pseudocode):

    Code:
    // set counter = 1
    // do the loop
        // check if the counter is equal to 3
            // if it is 3, reset it to 0 and call sleep( 1000 ) to sleep for 1 second
            
        // counter++ (increment the counter)
        // do your cfhttp call
    
    // end loop
    Last edited by kiteless; July 12th, 2013 at 08:38 AM.

IMN logo majestic logo threadwatch logo seochat tools logo