r - How to write to csv files using specific column data from txt files using loop in python?

Question

I have 167 (genes) X 26 (population) files with data as follows in each file. (Number of columns in each file is unknown)

CHROM POS PI 3 42809473 0 3 42809498 0.0104167 3 42809524 0 3 42809625 0 3 42809638 0 3 42809715 0.306283 3 42809846 0 3 42809910 0 3 42809911 0 3 42809964 0.306283 3 42810034 0.306283 3 42810082 0 3 42810098 0 3 42810103 0 3 42810184 0 3 42810189 0.306283

I wanted to compile data from 26 populations for each gene so as to obtain a single csv file. So, in total i need 167 csv files With 2nd column (see above) as header and 4th column (from each of 26 population files) in 26 rows.

Below is a sample csv file which I have made for 1 gene only.

Nucleotide_Position 42809473 42809498 42809524 42809625 AFR_ACB 0 0.01042 0 0 AFR_ASW 0 0 0 0 AFR_ESN 0 0.0201 0 0 AFR_GWD 0 0.00885 0 0 AFR_LWK 0 0 0.0201 0 AFR_MSL 0 0.03488 0 0 AFR_YRI 0 0.00926 0.00926 0 AMR_CLM 0 0 0 0.08192 AMR_MXL 0 0 0 0.01563 AMR_PEL 0 0 0 0.02339 AMR_PUR 0 0 0 0.02857 EAS_CDX 0 0 0 0 EAS_CHB 0.00971 0 0 0 EAS_CHS 0 0 0 0 EAS_JPT 0 0 0 0.01914 EAS_KHV 0 0 0 0.0101 EUR_CEU 0 0 0 0.19059

Below is my coding. Everything works well when I use a print statement and view the results on terminal. However, I am unable to write the 26 rows (PI) into separate csv files. I have tried changing the indentation of the write statement but nothing seems to work.

file_1 = open('/home/aahm/Desktop/Sitepi_Analysis/pop_list.txt','r')
a = file_1.readline()
b = a.strip('
')
pop_list = b.split(',')
pop_list.sort()


file_2 = open('/home/aahm/Desktop/Sitepi_Analysis/gene_list.txt','r')
c = file_2.readline()
d = c.strip('
')
gene_list = d.split(',')
gene_list.sort()


for gene_Name in gene_list:
    e = '/home/aahm/Desktop/Sitepi_Analysis/'+gene_Name+'/1k_AFR_ACB_'+gene_Name+'.sites.pi'
    f = '/home/aahm/Desktop/Sitepi_Analysis/csv_files/'+'sitepi_'+gene_Name+'.csv'
    file_3 = open(e,'r')
    file_4 = open(f,'w')
    file_3_data = file_3.readlines()
    nuc_pos_list = []
    for lines in file_3_data:
        remove_newline = lines.strip('
')
        line_list = remove_newline.split('') 
        nuc_pos = line_list[1]
        nuc_pos_list.append(nuc_pos)
        nuc_pos_list[0] = 'Nucleotide Position'
        nuc_pos_str = ','.join(nuc_pos_list)
        header = nuc_pos_str+'
'
    file_4.write(header)
    

for gene_Name in gene_list:
    PI_list_allpops = []
    for pop_Name in pop_list:
        g = '/home/aahm/Desktop/Sitepi_Analysis/'+gene_Name+'/1k_'+pop_Name+'_'+gene_Name+'.sites.pi'
        file_5 = open(g,'r')
        file_5_data = file_5.readlines()
        PI_list = []
        for lines in file_5_data:
            remove_newline = lines.strip('
')
            line_list = remove_newline.split('') 
            PI = line_list[-1]
            PI_list.append(PI)
        PI_list_allpops.append(PI_list)     
    for i in range(len(PI_list_allpops)):
        j = PI_list_allpops[i]
        j[0] = pop_list[i]
    for PI_values_list in PI_list_allpops:
        for PI_values in PI_values_list:
            PI_values_str = ','.join(PI_values_list)
            PI_values_string = str(PI_values_str)
            final_str = PI_values_string + '
'
        file_4.write(final_str)


file_1.close()
file_2.close()
file_3.close()
file_4.close()
file_5.close()

I have been struggling to get these results and I am stuck with the last bit. I would highly appreciate any help. Thanks a lot in advance! Maybe there is a better way of doing it too. I need the data for analysis with R, but since I am more familiar with python than R, I thought of converting it in python first. I'm new to coding and it's my first time using R. Is it possible to attach some files?

Answer 1

等待大神答复