Read Collection or Sequence of Text Files

When your data is stored across multiple text files, you can use tabularTextDatastore to manage and import the data. This example shows how to use tabularTextDatastore to read the data from the collection of text files all together, or to read one file at a time.

Data

For this example, the folder C:\DataTxt contains a collection of text files. Capture this location in the variable location. The data contains 10 text files, where each file contains 10 rows of data. The results differ based on your files and data.

location = 'C:\DataTxt';
dir(location)

.           File01.csv  File03.csv  File05.csv  File07.csv  File09.csv  
..          File02.csv  File04.csv  File06.csv  File08.csv  File10.csv  

Create Datastore

Create a datastore using the location of the files.

ds = tabularTextDatastore(location)

ds = 
  TabularTextDatastore with properties:

                      Files: {
                             'C:\DataTxt\File01.csv';
                             'C:\DataTxt\File02.csv';
                             'C:\DataTxt\File03.csv'
                              ... and 7 more
                             }
               FileEncoding: 'UTF-8'
   AlternateFileSystemRoots: {}
          ReadVariableNames: true
              VariableNames: {'LastName', 'Gender', 'Age' ... and 7 more}
             DatetimeLocale: en_US

  Text Format Properties:
             NumHeaderLines: 0
                  Delimiter: ','
               RowDelimiter: '\r\n'
             TreatAsMissing: ''
               MissingValue: NaN

  Advanced Text Format Properties:
            TextscanFormats: {'%q', '%q', '%f' ... and 7 more}
                   TextType: 'char'
         ExponentCharacters: 'eEdD'
               CommentStyle: ''
                 Whitespace: ' \b\t'
    MultipleDelimitersAsOne: false

  Properties that control the table returned by preview, read, readall:
      SelectedVariableNames: {'LastName', 'Gender', 'Age' ... and 7 more}
            SelectedFormats: {'%q', '%q', '%f' ... and 7 more}
                   ReadSize: 20000 rows

Read Data from Datastore

Use the read or readall functions to import the data from the datastore. If the data from the collection fits in the memory, you can import it all at once using the readall function.

allData = readall(ds);
size(allData)

ans = 1×2

   100    10

Alternatively, import the data one file at a time using the read function. To control the amount of data imported, before you call read, adjust the ReadSize property of the datastore. Set the ReadSize to 'file' or a positive integer.

• If ReadSize is 'file', then each call to read reads all the data one file at a time.

• If ReadSize is a positive integer, then each call to read reads the number of rows specified by ReadSize, or fewer, if it reaches the end of the data.

ds.ReadSize = 'file';
firstFile = read(ds) % reads first file

firstFile=10×10 table
     LastName      Gender     Age             Location              Height    Weight    Smoker     Systolic    Diastolic    SelfAssessedHealthStatus
    __________    ________    ___    ___________________________    ______    ______    _______    ________    _________    ________________________

    'Smith'       'Male'      38     'County General Hospital'        71       176      'TRUE'       124          93              'Excellent'       
    'Johnson'     'Male'      43     'VA Hospital'                    69       163      'FALSE'      109          77              'Fair'            
    'Williams'    'Female'    38     'St. Mary's Medical Center'      64       131      'FALSE'      125          83              'Good'            
    'Jones'       'Female'    40     'VA Hospital'                    67       133      'FALSE'      117          75              'Fair'            
    'Brown'       'Female'    49     'County General Hospital'        64       119      'FALSE'      122          80              'Good'            
    'Davis'       'Female'    46     'St. Mary's Medical Center'      68       142      'FALSE'      121          70              'Good'            
    'Miller'      'Female'    33     'VA Hospital'                    64       142      'TRUE'       130          88              'Good'            
    'Wilson'      'Male'      40     'VA Hospital'                    68       180      'FALSE'      115          82              'Good'            
    'Moore'       'Male'      28     'St. Mary's Medical Center'      68       183      'FALSE'      115          78              'Excellent'       
    'Taylor'      'Female'    31     'County General Hospital'        66       132      'FALSE'      118          86              'Excellent'       

secondFile = read(ds) % reads second file

secondFile=10×10 table
     LastName      Gender     Age             Location              Height    Weight    Smoker     Systolic    Diastolic    SelfAssessedHealthStatus
    __________    ________    ___    ___________________________    ______    ______    _______    ________    _________    ________________________

    'Anderson'    'Female'    45     'County General Hospital'        68       128      'FALSE'      114          77              'Excellent'       
    'Thomas'      'Female'    42     'St. Mary's Medical Center'      66       137      'FALSE'      115          68              'Poor'            
    'Jackson'     'Male'      25     'VA Hospital'                    71       174      'FALSE'      127          74              'Poor'            
    'White'       'Male'      39     'VA Hospital'                    72       202      'TRUE'       130          95              'Excellent'       
    'Harris'      'Female'    36     'St. Mary's Medical Center'      65       129      'FALSE'      114          79              'Good'            
    'Martin'      'Male'      48     'VA Hospital'                    71       181      'TRUE'       130          92              'Good'            
    'Thompson'    'Male'      32     'St. Mary's Medical Center'      69       191      'TRUE'       124          95              'Excellent'       
    'Garcia'      'Female'    27     'VA Hospital'                    69       131      'TRUE'       123          79              'Fair'            
    'Martinez'    'Male'      37     'County General Hospital'        70       179      'FALSE'      119          77              'Good'            
    'Robinson'    'Male'      50     'County General Hospital'        68       172      'FALSE'      125          76              'Good'