Can I generalize a folder lookup location so that data can be accessed in more than one way?

6 ビュー (過去 30 日間)
Ryan Andosca
Ryan Andosca 2023 年 8 月 9 日
編集済み: Ryan Andosca 2023 年 8 月 10 日
I'm finding this issue difficult to describe, so bear with me...
So a colleague created a structure that looks for files in specified location, for example if I call the structure.get_image() function, it looks in folder '/media/Server/user/...'
However, the data itself has been copied to different servers and is accessed from different workstations (sometimes with different operating systems), and not all of those workstations and servers have the same file structure. For example, in one it may need to look in '/data/DifferentServer/user...' - everything past the server name should be the same for everyone and every type of access, but not so with the server name itself and what comes before it. It may also need to look in a windows computer at a location like Z:\media\Server\user\...' which possibly poses a different problem.
So my question is, is there a way to generalize the folder location to work in all of these cases?
Structure example (omitted some things, but all of the omitted things are universal regardless of how the data is accessed):
algorithm: 'omitted'
parameters: [1×1 struct]
nJobs: 2
comment: []
folder: '/media/Cube/omitted/' % The problem
refScan: 1
uuid: (leaving this out)
patient: []
study: [1×1 study]
sliceFolder: '/media/Cube/omitted/omitted/' % Also the problem
% There is a lot more to these structures, but I think this gives the idea
Error received if accessed from a server that instead has the folder located at '/data/Sphere/omitted/':
Cannot find file
"/media/Cube/omitted/01_deformed.nii".
Currently people have to go in and do this if they want to use the data from another location:
set(thing.subThing,'folder','data/Sphere/omitted');
set(thing.subThing,'sliceFolder','data/Sphere/omitted');
% About 5 more things to set in addition to these, but you get the idea
And then they can't save those changes or else it may mess it up for other people accessing the data. And sometimes we want to work with dozens of these structures at once so we would have to do these sets for each one (and the "omitted" part is different for each one, so it can't be looped). Ideally the folder location would be instead something like:
folder: '.../omitted/'
% Same for sliceFolder and all the other paths
Where the "..." is determined based on where "omitted" is actually located for that user.
Is this sort of generalized pointer possible?
Apologies again for how difficult this has been for me to explain, please let me know if I can provide any additional clarification!
  6 件のコメント
Steven Lord
Steven Lord 2023 年 8 月 10 日
So to write paths that work on both, use /.
To write folder paths that work on Linux, Windows, and Mac use fullfile. Or if you can't for some reason, use filesep.
Ryan Andosca
Ryan Andosca 2023 年 8 月 10 日
編集済み: Ryan Andosca 2023 年 8 月 10 日
Going to be looking into these solutions today, but to answer some questions and provide more information:
The users are fairly adept users, but they already have a bunch of code that uses this data. They want to be able to access the data from our new server that we uploaded the data to, but the location the data is stored in in the new server (call it Sphere) doesn't match the old server (call it Cube). So I'm trying to provide a way that users can run their code on the new server and have it work out-of-the-box so to speak, rather than them having to do something like alter the folder locations themselves.
I wrote a very simple function based on an earlier suggestion that users can run when they open up the data that will update the folder locations locally (since some people still use the data on the old server) that people can use for now, but if we have new people come in to work with this data it would be easier if it didn't require they run a function based on how they're accessing the data (I wrote one for each server that has the data). Though I get it if that's just what is required.
Appreciate all of the responses and I'll be looking into each one more today!

サインインしてコメントする。

回答 (1 件)

Jan
Jan 2023 年 8 月 10 日
The core of the problem is a messed up storage of files in a bunch of folders. Collecting the files in one shared folder would be much cleaner. A database is even better to store a bunch of data.
It is not hard to write a workaround: The folders needed by a user are defined in a function e.g. called "PersonalDataFolder" in the personal user path. Then calling the sharded function "FindFile" requests the list of the folders and searchs for the specific file in the set of given folders:
function File = FindFile(FileName)
FolderList = PersonalDataFolder(); % E.g. {'/media/Server1/user1', '/media/server2/user1'}
File = '$File not found$';
for iFolder = 1:numel(FolderList)
aFile = fullfile(FolderList{iFolder}, FileName);
if isfile(aFile)
File = aFile;
break;
end
% Alternatively: Check the other folders also to find ambiguities
end
end
  1 件のコメント
Ryan Andosca
Ryan Andosca 2023 年 8 月 10 日
編集済み: Ryan Andosca 2023 年 8 月 10 日
I'm concerned that I'm not understanding this solution correctly, though I'd like to try it.
So each object (aThing) has a bunch of data associated with it (aThing.get_data, aThing.get_otherData, aThing.subThing.get_data, etc) and each of those "get_data" functions look for that data in different folders associated with that object instance. When I run a get_data on a place other than the original server, it looks in the "wrong" folder as described. Is the suggestion that insead of delineating folders for each object instance, all the data should be stored in a single folder and the proposed FindFile function should be used in "get_data" to pull the correct file?
Worth mentioning I didn't make these objects and I sort of inherited this issue. I just use the data the most, so it's become my job to ensure it's accessible from any of the servers that it is stored on. I'm an intermediate MATLAB user at best!

サインインしてコメントする。

カテゴリ

Help Center および File ExchangeStartup and Shutdown についてさらに検索

製品


リリース

R2023a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by