Data store only importing 2 columns

Hey,

I am trying to do a large data analysis using datastore but when trying to import multiple .csv files instead of splitting the data into multiple columns for each variable it is only making 2 columns. The first column being the first variable and second colum being all the other data sperated by commas. How do I tell the data store to seperate the other data into columns? I have attatched two example .csv.

the script I am using is simply;

ds = datastore('FileLocation\*.csv')

Thanka for the help!

Answer 1

[dpb]

DateTimeUTC,AIMHeading,AccX,AccY,AccZ,Surge,Sway,Heave,Roll,Pitch,Inclination,Yaw,HeaveAtRadar,RadarValue,CompensatedRadarValue,AvgWaveHeight,MaxWaveHeight,SignWaveHeight,SignWavePeriod,MeanWavePeriod,PeakPeriod,AvgWaveDir,AvgWaveSpread
2019-08-09 07.01.01,228.9000,0.3864,0.3124,0.0544,0.2421,0.3638,0.0059,-1.5691,-0.3287,1.6032,0.1163,0.0090,-7.5910,7.6000,0.4000,0.9500,0.5900,8.1000,6.6000,5.8000,0.0000,0.0000
2019-08-09 07.01.02,228.9000,-0.1636,-0.0300,-0.0195,-0.18 ...

The problem is the one see so often -- the semi-broke .csv file with embedded blanks in a field without quoted strings.

However, looks like

ds=datastore('FileLocation\*.csv','delimiter',{','});

does the trick...at least for one file; I didn't download both.

>> ds=datastore('201908090801AIM1_1H.csv',"Delimiter",{','})
ds = 
  TabularTextDatastore with properties:

                      Files: {
                             'C:\Users\Duane\Documents\MATLAB\Work\201908090801AIM1_1H.csv'
                             }
               FileEncoding: 'UTF-8'
   AlternateFileSystemRoots: {}
      PreserveVariableNames: false
          ReadVariableNames: true
              VariableNames: {'DateTimeUTC', 'AIMHeading', 'AccX' ... and 20 more}
             DatetimeLocale: en_US

  Text Format Properties:
             NumHeaderLines: 0
                  Delimiter: {','}
               RowDelimiter: '\r\n'
             TreatAsMissing: ''
               MissingValue: NaN

  Advanced Text Format Properties:
            TextscanFormats: {'%q', '%f', '%f' ... and 20 more}
                   TextType: 'char'
         ExponentCharacters: 'eEdD'
               CommentStyle: ''
                 Whitespace: ' \b\t'
    MultipleDelimitersAsOne: false

  Properties that control the table returned by preview, read, readall:
      SelectedVariableNames: {'DateTimeUTC', 'AIMHeading', 'AccX' ... and 20 more}
            SelectedFormats: {'%q', '%f', '%f' ... and 20 more}
                   ReadSize: 20000 rows

>> preview(ds)
ans =
  8×23 table
          DateTimeUTC          AIMHeading     AccX       AccY       AccZ       Surge      Sway       Heave      Roll       Pitch     Inclination      Yaw      HeaveAtRadar    RadarValue    CompensatedRadarValue    AvgWaveHeight    MaxWaveHeight    SignWaveHeight    SignWavePeriod    MeanWavePeriod    PeakPeriod    AvgWaveDir    AvgWaveSpread
    _______________________    __________    _______    _______    _______    _______    _______    _______    _______    _______    ___________    _______    ____________    __________    _____________________    _____________    _____________    ______________    ______________    ______________    __________    __________    _____________
    {'2019-08-09 07.01.01'}      228.9        0.3864     0.3124     0.0544     0.2421     0.3638     0.0059    -1.5691    -0.3287      1.6032        0.1163        0.009         -7.591                7.6                 0.4             0.95              0.59              8.1               6.6             5.8            0               0      
    {'2019-08-09 07.01.02'}      228.9       -0.1636      -0.03    -0.0195    -0.1808     0.1131    -0.0466    -1.9137     0.0113      1.9137        0.1565        0.124         -7.592              7.716                 0.4             0.95              0.59              8.1               6.6             5.8            0               0      
    {'2019-08-09 07.01.03'}      229.1       -0.2974    -0.2663    -0.0237    -0.3529    -0.0107    -0.0645     -1.299    -1.0392      1.6635       -0.0378        0.097         -7.592              7.689                 0.4             0.95              0.59              8.1               6.6             5.8            0               0      
    {'2019-08-09 07.01.04'}      229.3       -0.1462    -0.1184       0.04    -0.2197     0.1167    -0.0527    -0.6783    -2.0269      2.1373        0.0396       -0.096         -7.591              7.495                 0.4             0.95              0.59              8.1               6.6             5.8            0               0      
    {'2019-08-09 07.01.05'}      229.4        0.0183     0.2937     0.0223     -0.029     0.2389    -0.0537    -1.0576    -2.0764      2.3301        0.1158       -0.268         -7.591              7.323                 0.4             0.95               0.6              8.4               6.6             5.8            0               0      
    {'2019-08-09 07.01.06'}      229.4        0.3044     0.0866    -0.0578    -0.0044     0.0772    -0.0475    -1.7729    -1.1997      2.1406        0.0002       -0.206         -7.591              7.385                 0.4             0.95              0.59              8.4               6.6             5.8            0               0      
    {'2019-08-09 07.01.07'}      229.4        0.0308    -0.1888    -0.0367    -0.2413    -0.1377      0.008    -1.6663    -0.7888      1.8435       -0.1285        0.013          -7.59              7.603                 0.4             0.95              0.59              8.1               6.5             5.8            0               0      
    {'2019-08-09 07.01.08'}      229.3       -0.3493    -0.1879     0.0397    -0.4157    -0.1401      0.076    -0.9687    -1.3592       1.669       -0.0366        0.167         -7.591              7.758                 0.4             0.95              0.59              8.2               6.6             5.8            0               0      
>> 

[dpb]

I'll note that that file was "only" 3600 lines -- that's easily handled in memory so not sure you need the datastore at all here -- unless going to read in the multiple files together. Not sure if that's how it works or not...never used enough to know.

[Andrew Green]

It seems to work, cheers for the help :)