1. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2013
    Rep Power

    For Loop not Iterating over Files in Directory

    I am trying to get the first for loop below "for fn in dirs:" to iterate over a list of files in a directory and merge the ones with similar filenames. The loop seems to only execute once, instead of for all files listed in "dirs = os.listdir(path)". I have tried several things mentioned on other posts but nothing has worked.

    Can you help me understand why this for loop is not iterating?

        import pandas as pd
        import os
        import glob 
        path = r"C:\Users\St\Documents\House\m2"
        dirs = os.listdir(path)  #list of .csv files
        for fn in dirs:
            for fn in glob.glob(os.path.join(path, '*2.csv')):
                W = fn      
                if len(W) == 48:   # some filenames are one character shorter
                    root2 = W[:43]
                    root2 = W[:42]
            for fn in glob.glob(os.path.join(path, '*J.csv')):
                J = fn
                if len(J) == 48:
                    jroot = J[:43]
                    jroot = J[:42]
            for fn in glob.glob(os.path.join(path, '*T.csv')):
                T = fn      
                if len(T) == 48:
                    troot = T[:43]
                    troot = T[:42]         
            for fn in glob.glob(os.path.join(path, '*S.csv')):
                S = fn      
                if len(S) == 48:
                    sroot = S[:43]
                    sroot = S[:42]              
            for fn in glob.glob(os.path.join(path, '*D.csv')):
                D = fn     
                if len(D) == 48:
                    droot = D[:43]
                    droot = D[:42] 
            for fn in glob.glob(os.path.join(path, '*R.csv')):
                R = fn       
                if len(R) == 48:               
                    rroot = R[:43]
                    rroot = R[:42]       
            WW = pd.read_csv(W) #read csv so I can use pd.merge
            JJ = pd.read_csv(J)   
            TT = pd.read_csv(T)
            SS = pd.read_csv(S) 
            DD = pd.read_csv(D)
            RR = pd.read_csv(R)
            if jroot == root2:  #merge files with names that start with same characters
                JW = pd.merge(WW, JJ, on="JN")
            if jroot == troot:
                TW = pd.merge(JW, TT, on="TN")
            if jroot == sroot:
                SW = pd.merge(TW, SS, on="SN")   
            if jroot == droot:
                DW = pd.merge(SW, DD, on="DN")
            if jroot == rroot:
                RW = pd.merge(RR, DW, on="HN")
                data = RW
                data.to_csv(jroot + "x.csv")  #export final merged dataframe to .csv
  2. #2
  3. Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Feb 2005
    Rep Power
    Mixing in a few temporary print() might help.
    Real Programmers always confuse Christmas and Halloween because Oct31 == Dec25

IMN logo majestic logo threadwatch logo seochat tools logo