# Running on many samples

To run the pipeline on a few samples if relatively straightforward, but when we need to run it on 100s of samples it can get a bit unwieldy to type every command out. For this purpose it is a good idea to write a wrapper script to handle the running.

As an example let's assume that you have the following files:

```
.
└── fastq
    ├── ERR1664623_1.fastq.gz
    ├── ERR1664623_2.fastq.gz
    ├── ERR1664624_1.fastq.gz
    ├── ERR1664624_2.fastq.gz
    ├── ERR1664635_1.fastq.gz
    ├── ERR1664635_2.fastq.gz
    ├── ERR1664636_1.fastq.gz
    ├── ERR1664636_2.fastq.gz
    ├── ERR1664664_1.fastq.gz
    ├── ERR1664664_2.fastq.gz
    ├── ERR1664665_1.fastq.gz
    └── ERR1664665_2.fastq.gz
```

First we will need to create a list of the samples prefixes that you would like to run. To do this we can run the following command.

```
ls fastq/ | grep _1.fastq.gz | sed 's/_1.fastq.gz//' > samples.txt
```

This will put all the file prefixes into a file called samples.txt. We can then use parallel to run our tb-profiler command for each sample in the file like this. Before we run tb-profiler we should make the folders where it will store the bam vcf and result files. We have to do this because ottherwise the multiple instances of tb-profiler run by parallel will all try to create the same folders at the same time and you will run into an error.

```
mkdir bam vcf results
```

Now we are ready to run tb-profiler in parallel.

```
cat samples.txt | parallel --bar -j 2 tb-profiler profile -1 fastq/{}_1.fastq.gz -2 fastq/{}_2.fastq.gz -p {}
```

You can adjust the `-j` parameter to allow for more jobs to run in parallel. I have set this to **2** but if you have a HPC or powerful computer you can increase this.