#1
  1. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2013
    Posts
    3
    Rep Power
    0

    For Loop not Iterating over Files in Directory


    I am trying to get the first for loop below "for fn in dirs:" to iterate over a list of files in a directory and merge the ones with similar filenames. The loop seems to only execute once, instead of for all files listed in "dirs = os.listdir(path)". I have tried several things mentioned on other posts but nothing has worked.

    Can you help me understand why this for loop is not iterating?

    Code:
        import pandas as pd
        import os
        import glob 
        
        path = r"C:\Users\St\Documents\House\m2"
        dirs = os.listdir(path)  #list of .csv files
        
        for fn in dirs:
            for fn in glob.glob(os.path.join(path, '*2.csv')):
                W = fn      
                if len(W) == 48:   # some filenames are one character shorter
                    root2 = W[:43]
                else:
                    root2 = W[:42]
            for fn in glob.glob(os.path.join(path, '*J.csv')):
                J = fn
                if len(J) == 48:
                    jroot = J[:43]
                else:
                    jroot = J[:42]
            for fn in glob.glob(os.path.join(path, '*T.csv')):
                T = fn      
                if len(T) == 48:
                    troot = T[:43]
                else:
                    troot = T[:42]         
            for fn in glob.glob(os.path.join(path, '*S.csv')):
                S = fn      
                if len(S) == 48:
                    sroot = S[:43]
                else:
                    sroot = S[:42]              
            for fn in glob.glob(os.path.join(path, '*D.csv')):
                D = fn     
                if len(D) == 48:
                    droot = D[:43]
                else:
                    droot = D[:42] 
            for fn in glob.glob(os.path.join(path, '*R.csv')):
                R = fn       
                if len(R) == 48:               
                    rroot = R[:43]
                else:
                    rroot = R[:42]       
                    
            WW = pd.read_csv(W) #read csv so I can use pd.merge
            JJ = pd.read_csv(J)   
            TT = pd.read_csv(T)
            SS = pd.read_csv(S) 
            DD = pd.read_csv(D)
            RR = pd.read_csv(R)
                    
            if jroot == root2:  #merge files with names that start with same characters
                JW = pd.merge(WW, JJ, on="JN")
            if jroot == troot:
                TW = pd.merge(JW, TT, on="TN")
            if jroot == sroot:
                SW = pd.merge(TW, SS, on="SN")   
            if jroot == droot:
                DW = pd.merge(SW, DD, on="DN")
            if jroot == rroot:
                RW = pd.merge(RR, DW, on="HN")
                data = RW
                data.to_csv(jroot + "x.csv")  #export final merged dataframe to .csv
  2. #2
  3. Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Feb 2005
    Posts
    605
    Rep Power
    65
    Mixing in a few temporary print() might help.
    Real Programmers always confuse Christmas and Halloween because Oct31 == Dec25

IMN logo majestic logo threadwatch logo seochat tools logo