#1
  1. Mad Scientist
    Devshed Expert (3500 - 3999 posts)

    Join Date
    Oct 2007
    Location
    North Yorkshire, UK
    Posts
    3,660
    Rep Power
    4123

    PDO, Iterators and a bit of confusion


    Having seen the news that PHP 5.5 is in alpha release I went to see what was new and saw a term I was unfamiliar with - "generators", and ended up reading this RFC. I could immediately see the potential for reducing memory and looping (assuming that the binding of functions and use of yield does not introduce more over head than using a full dataset or looping twice*)....so I started to look at the Iterator examples the RFC gives as a current alternative...and this is where I started to get confused with what to do and what is actually going on.

    The example is with respect to reading files from the filesystem. A handler is opened by the Iterator class and, as I understand it, only returns the current line in question.

    This got me thinking about how my app works with my database, and the database abstraction layer.

    My "getMany()" method in my models executes the fetchAll method of a PDOStatement object, thus returning the entire dataset as a 2D array, which is then often worked with again (eg in some views I loop over the array to create html tables and lists)

    So, I'm wondering if there's a way (in PHP < 5.5, ie without generators) to use the Iterator patterns to only have to loop over the data once - or at the very least not have put the entire database into memory (eg just use fetch, working on one row at a time)

    Now, the manual says that PDOStatement implements traversable.....but I've yet to find the complete way through.....

    any ideas or pointers?

    *when I say looping twice I do not mean nested looping, but looping over an array-like structure once to get some other array-like object, then looping over the original or new array later on the the script execution
    I said I didn't like ORM!!! <?php $this->model->update($this->request->resources[0])->set($this->request->getData())->getData('count'); ?>

    PDO vs mysql_* functions: Find a Migration Guide Here

    [ Xeneco - T'interweb Development ] - [ Are you a Help Vampire? ] - [ Read The manual! ] - [ W3 methods - GET, POST, etc ] - [ Web Design Hell ]
  2. #2
  3. Sarcky
    Devshed Supreme Being (6500+ posts)

    Join Date
    Oct 2006
    Location
    Pennsylvania, USA
    Posts
    10,690
    Rep Power
    6351
    If it implements Traversable, then it's an iterator. You can use it just like you'd use any other iterator. In fact, there's a whole set of functions that work only on Iterators. You'd probably love iterator_apply.
    HEY! YOU! Read the New User Guide and Forum Rules

    "They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -Benjamin Franklin

    "The greatest tragedy of this changing society is that people who never knew what it was like before will simply assume that this is the way things are supposed to be." -2600 Magazine, Fall 2002

    Think we're being rude? Maybe you asked a bad question or you're a Help Vampire. Trying to argue intelligently? Please read this.
  4. #3
  5. Mad Scientist
    Devshed Expert (3500 - 3999 posts)

    Join Date
    Oct 2007
    Location
    North Yorkshire, UK
    Posts
    3,660
    Rep Power
    4123
    This is the bit I really don't get...is where to implement the iterator and where to define the necessary methods where required

    For example, I'm trying to re-factor this code to make use of iterators:

    PHP Code:
    //

    class DB {

        
    /*...*/

        
    public function returnArray() {
            return 
    $this->SQL->fetchAll(\PDO::FETCH_ASSOC);
        }
        
        
    /*...*/
    }

    abstract class 
    model {

        
    /*...*/

        
    public function getMany();
            return 
    DB::Load()->Execute($this->sql,$this->args)->returnArray();
        }

        
    /*...*/
    }

    class 
    someObject extends model {

        
    /*...*/

        
    someFunction() {
            
    $data $this->getMany();
            foreach(
    $data as $key => $val) {
                
    /*...*/
            
    }
        }

        
    /*...*/


    I said I didn't like ORM!!! <?php $this->model->update($this->request->resources[0])->set($this->request->getData())->getData('count'); ?>

    PDO vs mysql_* functions: Find a Migration Guide Here

    [ Xeneco - T'interweb Development ] - [ Are you a Help Vampire? ] - [ Read The manual! ] - [ W3 methods - GET, POST, etc ] - [ Web Design Hell ]
  6. #4
  7. --
    Devshed Expert (3500 - 3999 posts)

    Join Date
    Jul 2012
    Posts
    3,910
    Rep Power
    1045
    Hi,

    I'm not sure what you expect from iterators in the context of queries, because query objects already are external iterators by means of their "fetch" method -- you just cannot rewind them (which is a general database issue and will apply any iterator wrapping the query stuff).

    The only advantage of an explicit iterator is that you can use a foreach loop instead of
    PHP Code:
    while ($row $stmt->fetch()) ... 
    If this is really that important to you, just replace DB::returnArray() with a method which returns the actual PDO statement. The getMany() method will then return the specific statement and allow the caller to iterate over the result set.

    You don't need to define your own iterator class, because PDOStatement already has everything you need.
  8. #5
  9. Sarcky
    Devshed Supreme Being (6500+ posts)

    Join Date
    Oct 2006
    Location
    Pennsylvania, USA
    Posts
    10,690
    Rep Power
    6351
    You already have an iterator. You could just return it and use it one level higher than you're using it now. Instead of calling FetchAll on it (or whatever), just return it and then have your code above this foreach over it.

    Of course, that stops abstraction from being...abstracted. You could also make a resultset object which itself implements either ArrayAccess/ArrayIterator or Traversable and then wrap the PDO result in THAT so that you have a custom wrapper library.

    Using the PDO iterator directly goes against the concept of database abstraction. The reason you're using fetchAll is because you don't want your code outside your database library to have to deal with database-specific code (though PDO is, itself, an abstraction library).

    To try to simplify this post:
    Your getMany() function right now returns an array. You can either:
    1) Have it return the iterator, use it in someObject

    2) Wrap the iterator in another iterator of your own design, use THAT in someObject

    3) Move on from this problem because very few people do it this way anyway, use iterators for something not so central and complicated.
    HEY! YOU! Read the New User Guide and Forum Rules

    "They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -Benjamin Franklin

    "The greatest tragedy of this changing society is that people who never knew what it was like before will simply assume that this is the way things are supposed to be." -2600 Magazine, Fall 2002

    Think we're being rude? Maybe you asked a bad question or you're a Help Vampire. Trying to argue intelligently? Please read this.
  10. #6
  11. Mad Scientist
    Devshed Expert (3500 - 3999 posts)

    Join Date
    Oct 2007
    Location
    North Yorkshire, UK
    Posts
    3,660
    Rep Power
    4123
    Originally Posted by ManiacDan
    To try to simplify this post:
    Your getMany() function right now returns an array. You can either:
    1) Have it return the iterator, use it in someObject

    2) Wrap the iterator in another iterator of your own design, use THAT in someObject

    3) Move on from this problem because very few people do it this way anyway, use iterators for something not so central and complicated.
    Okay, I hear you.

    Really, I'm looking for performance increases - which, I admit, I could find bigger savings elsewhere in my app but this whole iterators/generators is new to me and got me interested...and confused, but the confusion is clearing
    I said I didn't like ORM!!! <?php $this->model->update($this->request->resources[0])->set($this->request->getData())->getData('count'); ?>

    PDO vs mysql_* functions: Find a Migration Guide Here

    [ Xeneco - T'interweb Development ] - [ Are you a Help Vampire? ] - [ Read The manual! ] - [ W3 methods - GET, POST, etc ] - [ Web Design Hell ]
  12. #7
  13. Sarcky
    Devshed Supreme Being (6500+ posts)

    Join Date
    Oct 2006
    Location
    Pennsylvania, USA
    Posts
    10,690
    Rep Power
    6351
    The best way to get performance boosts is to change all your code to use the PDO iterator directly instead of using the array, because as you said that requires a double-loop through the data.

    You'd get a much bigger performance boost by researching and installing a caching/accelerator program.
    HEY! YOU! Read the New User Guide and Forum Rules

    "They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -Benjamin Franklin

    "The greatest tragedy of this changing society is that people who never knew what it was like before will simply assume that this is the way things are supposed to be." -2600 Magazine, Fall 2002

    Think we're being rude? Maybe you asked a bad question or you're a Help Vampire. Trying to argue intelligently? Please read this.
  14. #8
  15. Mad Scientist
    Devshed Expert (3500 - 3999 posts)

    Join Date
    Oct 2007
    Location
    North Yorkshire, UK
    Posts
    3,660
    Rep Power
    4123
    Originally Posted by ManiacDan
    You'd get a much bigger performance boost by researching and installing a caching/accelerator program.
    accelerator yes, caching - will have to be careful as about 50% of the data is actively changing and really does not want to be stale
    I said I didn't like ORM!!! <?php $this->model->update($this->request->resources[0])->set($this->request->getData())->getData('count'); ?>

    PDO vs mysql_* functions: Find a Migration Guide Here

    [ Xeneco - T'interweb Development ] - [ Are you a Help Vampire? ] - [ Read The manual! ] - [ W3 methods - GET, POST, etc ] - [ Web Design Hell ]
  16. #9
  17. Come play with me!
    Devshed Supreme Being (6500+ posts)

    Join Date
    Mar 2007
    Location
    Washington, USA
    Posts
    13,744
    Rep Power
    9397
    Also worth mentioning: IteratorAggregate. Lets you define a method that returns an iterator to use, so if your class wraps an array and you want to iterate over that, return an ArrayIterator($this->wrapped_array) - don't have to implement all the Iterator methods.

IMN logo majestic logo threadwatch logo seochat tools logo