Update the MRI section, fix typos #14
10 changed files with 32 additions and 116 deletions
|
|
@ -22,7 +22,7 @@ However, the service could also run under any other (existing) user account.
|
||||||
# new user, prohibit login, disable password
|
# new user, prohibit login, disable password
|
||||||
sudo adduser dumpthing --disabled-password --disabled-login
|
sudo adduser dumpthing --disabled-password --disabled-login
|
||||||
|
|
||||||
# allow this user to run prcoess while not logged in
|
# allow this user to run process while not logged in
|
||||||
sudo loginctl enable-linger dumpthing
|
sudo loginctl enable-linger dumpthing
|
||||||
|
|
||||||
# allow this user to execute systemd commands interactively.
|
# allow this user to execute systemd commands interactively.
|
||||||
|
|
|
||||||
|
|
@ -39,7 +39,7 @@ NeuroBagel will be deployed under this user account, and all software and data w
|
||||||
# new user, prohibit login, disable password
|
# new user, prohibit login, disable password
|
||||||
sudo adduser neurobagel --disabled-password --disabled-login
|
sudo adduser neurobagel --disabled-password --disabled-login
|
||||||
|
|
||||||
# allow this user to run prcoess while not logged in
|
# allow this user to run process while not logged in
|
||||||
sudo loginctl enable-linger neurobagel
|
sudo loginctl enable-linger neurobagel
|
||||||
|
|
||||||
# allow this user to execute systemd commands interactively.
|
# allow this user to execute systemd commands interactively.
|
||||||
|
|
|
||||||
|
|
@ -32,7 +32,7 @@ Enter just the DOI without the URL project (i.e., `10.1016/j.neubiorev.2025.1063
|
||||||
|
|
||||||
7. Navigate to your new Publication record in the pool's Publication records list, and click the pencil icon to edit it.
|
7. Navigate to your new Publication record in the pool's Publication records list, and click the pencil icon to edit it.
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
8. For publications to make the best impressions and live up to their fullest metadata potential, proceed to add relevant information to complete the Publication record as follows:
|
8. For publications to make the best impressions and live up to their fullest metadata potential, proceed to add relevant information to complete the Publication record as follows:
|
||||||
|
|
||||||
|
|
@ -74,7 +74,7 @@ Enter just the DOI without the URL project (i.e., `10.1016/j.neubiorev.2025.1063
|
||||||
|
|
||||||
{{% /expand %}}
|
{{% /expand %}}
|
||||||
|
|
||||||
{{% expand title="Funding acknowledgement" %}}
|
{{% expand title="Funding acknowledgment" %}}
|
||||||
|
|
||||||
- At the very bottom of your Publication record editing screen, toggle the "All fields" option to on (indicated in blue).
|
- At the very bottom of your Publication record editing screen, toggle the "All fields" option to on (indicated in blue).
|
||||||
- Within the "Annotations" box, search "TRR funding" and select whether TRR funding is or is NOT acknowledged in the publication.
|
- Within the "Annotations" box, search "TRR funding" and select whether TRR funding is or is NOT acknowledged in the publication.
|
||||||
|
|
|
||||||
|
|
@ -1,45 +0,0 @@
|
||||||
---
|
|
||||||
title: Archive DICOMs
|
|
||||||
---
|
|
||||||
|
|
||||||
As the "true" raw data, DICOMs are rarely (re)accessed and hardly ever need to change.
|
|
||||||
However, they need to be stored somewhere.
|
|
||||||
Tracking DICOMs in DataLad datasets allows dependency tracking for conversion to NIfTI.
|
|
||||||
However, it's good to keep DataLad optional (allow DataLad and non-DataLad access).
|
|
||||||
|
|
||||||
## Historical precedent: ICF at FZJ
|
|
||||||
|
|
||||||
The following solution has been proposed for the Imaging Core Facility at FZJ:
|
|
||||||
|
|
||||||
- DICOMs are packed into tar files (tarballs) [^1]
|
|
||||||
- the tarballs are placed on a web server (intranet only), organized
|
|
||||||
by project (HTTP Basic Authentication for access management)
|
|
||||||
- DataLad datasets record availability via
|
|
||||||
[archivist](https://docs.datalad.org/projects/next/en/latest/generated/datalad_next.annexremotes.archivist.html)
|
|
||||||
and
|
|
||||||
[uncurl](https://docs.datalad.org/projects/next/en/latest/generated/datalad_next.annexremotes.uncurl.html)
|
|
||||||
special remotes, which translates to:
|
|
||||||
- a file is available from a tarball (archivist special remote)
|
|
||||||
- a tarball is available from a given URL, pointing to the web server (uncurl special remote)[^2].
|
|
||||||
- Only the Git repository (no annex) is stored by the consuming institute;
|
|
||||||
the ICF web server is the permanent DICOM storage.
|
|
||||||
|
|
||||||
The system has been documented in https://inm-icf-utilities.readthedocs.io/en/latest/
|
|
||||||
and the tarball & dataset generation tools implementation is in https://github.com/psychoinformatics-de/inm-icf-utilities.
|
|
||||||
|
|
||||||
[^1]: timestamps are normalized to ensure re-packing the same data does not change tarball checksums
|
|
||||||
[^2]: uncurl is chosen because it allows re-writing patterns with just configuration, e.g., should the base URL change
|
|
||||||
|
|
||||||
## TRR reimplementation
|
|
||||||
|
|
||||||
One of the TRR sites indicated intent to use a Forgejo instance for DICOM storage.
|
|
||||||
A particular challenge for the underlying system was [inode](https://en.wikipedia.org/wiki/Inode) limitation.
|
|
||||||
For this reason, an adaptation of the ICF system has been proposed:
|
|
||||||
|
|
||||||
- a dataset is generated upfront, and DICOM tarball is stored with the dataset in Forgejo (as annex)
|
|
||||||
- we keep using the archivist remote (file in tarball) to avoid using up thousands of inodes for individual files
|
|
||||||
(Git can pack its repository into several files, so we only add one more for the tarball).
|
|
||||||
|
|
||||||
A proof of principle for dataset generation (using re-written ICF
|
|
||||||
code) has been proposed in https://hub.trr379.de/q02/dicom-utilities. See the README for more
|
|
||||||
detailed explanations (and commit messages for even more detail).
|
|
||||||
|
|
@ -10,74 +10,35 @@ TRR379 uses the [Brain Imaging Data Structure](bids.neuroimaging.io) (BIDS) as t
|
||||||
## Conversion to BIDS
|
## Conversion to BIDS
|
||||||
|
|
||||||
The conversion of raw MRI data in DICOM format to a BIDS-compliant dataset is a largely automated process.
|
The conversion of raw MRI data in DICOM format to a BIDS-compliant dataset is a largely automated process.
|
||||||
The recommended software to be used for conversion is [Heudiconv](https://heudiconv.readthedocs.io).
|
The recommended software to be used for conversion is [heudiconv](https://heudiconv.readthedocs.io).
|
||||||
|
|
||||||
Heudiconv uses [dcm2niix](https://github.com/rordenlab/dcm2niix) as the actual DICOM→NIfTI converter.
|
Heudiconv uses [dcm2niix](https://github.com/rordenlab/dcm2niix) as the actual DICOM→NIfTI converter.
|
||||||
In our experience, `dcm2niix` is the most robust and most correct tool available for this task.
|
[Heudiconv tutorials](https://heudiconv.readthedocs.io/en/latest/tutorials.html) further illustrate how the software works.
|
||||||
|
|
||||||
Heudiconv does the job of mapping DICOM series to BIDS entities (ie. determine BIDS-compliant file names).
|
Heudiconv performs the task of mapping DICOM series to BIDS entities (ie. determining BIDS-compliant file names).
|
||||||
A key heudiconv concept is a **heuristic**: a Python program (function) which looks at the DICOM series properties and matches it with a file naming pattern.
|
A key heudiconv concept is a **heuristic**: a Python program (function) which receives the DICOM series properties and matches them with a file naming pattern.
|
||||||
A heuristic typically relies on DICOM series naming (set at the scanner console), but it can also use other properties such as number of images or acquisition parameters.
|
A heuristic typically relies on DICOM series naming (set at the scanner console), but it can also use other properties such as number of images or acquisition parameters.
|
||||||
|
|
||||||
Because TRR379 uses its own conventions, a matching heuristic needs to be provided (possibly one for each TRR379 site).
|
Heudiconv workflows have been implemented at each MRI acquisition site, and reuse shared components.
|
||||||
An [implementation of such a heuristic](https://hub.trr379.de/q02/phantom-mri-bids/src/branch/main/code/heuristic-q01.py) has been created, and was tested on phantom MRI acquisitions from all sites (see below).
|
|
||||||
Using this heuristic, MRI data from all sites can be BIDS-standardized.
|
|
||||||
As with any automation, caution and oversight is needed for edge cases (e.g. repeated / discarded acquisitions).
|
|
||||||
|
|
||||||
[Heudiconv tutorials](https://heudiconv.readthedocs.io/en/latest/tutorials.html) further illustrate the process and capabilities of the software.
|
## Shared Resources
|
||||||
|
|
||||||
### Good practices
|
- [q02/heudiconv-container](https://hub.trr379.de/q02/heudiconv-container) is a DataLad dataset providing a heudiconv container.
|
||||||
|
- [q02/rdmtools](https://hub.trr379.de/q02/rdmtools) contains Python scripts which further automate BIDS conversion, pushing created datasets to dedicated locations, and other data curation tasks.
|
||||||
|
- [q01/phantom-mri-dicoms](https://hub.trr379.de/q01/phantom-mri-dicoms) is a superdataset grouping phantom MRI scans from all sites, used to develop and test the conversion workflow; it also contains the [reference TRR379 heudiconv heuristic](https://hub.trr379.de/q01/phantom-mri-dicoms/src/branch/main/code/heuristic-q01.py).
|
||||||
|
|
||||||
- Use heudiconv as a containerized application. [Q02]({{% ref "/references/resources/projects/q02/" %}}) provides a [readily usable utility dataset with a
|
> [!important]
|
||||||
configured container](https://hub.trr379.de/q02/heudiconv-container). See that repository for
|
> Each site maintains *its own copy of the heuristic* (as part of the site-specific DICOM superdataset) to account for site-specific differences.
|
||||||
an example usage.
|
|
||||||
- DICOMs as subdatasets helps with provenance, even if those DICOMs are never accessed outside
|
|
||||||
- Heudiconv takes paths and (optionally) intended subject IDs as input
|
|
||||||
- if paths contain identifying information, this would leak into DataLad run records
|
|
||||||
- having a helper script / lookup table in the (private) DICOM dataset can hide this information
|
|
||||||
|
|
||||||
### Caveats
|
## Details
|
||||||
|
|
||||||
- https://hub.trr379.de/q02/phantom-mri-bids used dcm2niix v1.0.20240202
|
### Modular datasets
|
||||||
- current latest is v1.0.20250506
|
|
||||||
- potential impact discussed in https://hub.trr379.de/q02/phantom-mri-bids/issues/8
|
|
||||||
|
|
||||||
## Demonstrators and resources
|
A DataLad dataset containing a BIDS dataset will typically have these subdatasets.
|
||||||
|
Subdatasets help in recording provenance while enabling different access scopes for different components:
|
||||||
|
|
||||||
### TRR phantom DICOMs
|
- `code/heudiconv`: heudiconv container ([q02/heudiconv-container](https://hub.trr379.de/q02/heudiconv-container))
|
||||||
|
- `code/rdmtools`: optional, shared scripts for data curation tasks ([q02/rdmtools](https://hub.trr379.de/q02/rdmtools))
|
||||||
Scans of MRI phantoms were carried out using the intended sequences (presumably - see caveats section below).
|
- `sourcedata/dicoms`: a site-specific DICOM dataset, itself containing one subdataset per session
|
||||||
These were shared with Q02 and uploaded to the TRR Hub forgejo instance:
|
- `.heudiconv`: heudiconv cache / intermediate outputs are stored in a separate dataset to avoid including DICOM information in the BIDS dataset
|
||||||
|
|
||||||
- https://hub.trr379.de/q01/phantom-mri-dicom-aachen
|
|
||||||
- https://hub.trr379.de/q01/phantom-mri-dicom-frankfurt
|
|
||||||
- https://hub.trr379.de/q01/phantom-mri-dicom-heidelberg
|
|
||||||
- https://hub.trr379.de/q01/phantom-mri-dicom-mannheim
|
|
||||||
|
|
||||||
Note: Aachen did a re-scan which was shared by e-mail / cloud (June 03, 2025).
|
|
||||||
This has not been uploaded to forgejo (permissions + size).
|
|
||||||
|
|
||||||
### TRR phantom BIDS
|
|
||||||
|
|
||||||
- A BIDS-compliant dataset from these dicoms (3/4 sites): https://hub.trr379.de/q02/phantom-mri-bids
|
|
||||||
- The heuristic used: https://hub.trr379.de/q02/phantom-mri-bids/src/branch/main/code/heuristic-q01.py
|
|
||||||
- Issue tracker: https://hub.trr379.de/q02/phantom-mri-bids/issues
|
|
||||||
|
|
||||||
Conversion of re-scanned Aachen phantom is in https://hub.trr379.de/q02/tmp-phantom-bids (separate from the above
|
|
||||||
because input data is not available as a DataLad dataset)
|
|
||||||
|
|
||||||
### Data consistency
|
|
||||||
|
|
||||||
- the phantom datasets are not the same: https://hub.trr379.de/q02/phantom-mri-bids/issues/6
|
|
||||||
- re-scan from Aachen has more sequences than the initial scan, but lacks T2w
|
|
||||||
- heudiconv fails to parse some Heidelberg dicoms, and dcm2niix raises warnings;
|
|
||||||
unclear whether this is data issue or software issue: https://hub.trr379.de/q02/phantom-mri-bids/issues/5
|
|
||||||
|
|
||||||
### Conversion: technical issues
|
|
||||||
|
|
||||||
These are open questions:
|
|
||||||
|
|
||||||
- Technical: BIDS validator errors https://hub.trr379.de/q02/phantom-mri-bids/issues/7
|
|
||||||
- Technical: re-run with the latest dcm2niix https://hub.trr379.de/q02/phantom-mri-bids/issues/8
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -148,7 +148,7 @@ of TRR379.
|
||||||
Sites are free to implement any RDM solutions, as long as that infrastructure
|
Sites are free to implement any RDM solutions, as long as that infrastructure
|
||||||
provides
|
provides
|
||||||
|
|
||||||
- (programmatically) queriable metadata of a previously agreed upon nature
|
- (programmatically) queryable metadata of a previously agreed upon nature
|
||||||
- (programmatically) accessible data to any authorized members of TRR379
|
- (programmatically) accessible data to any authorized members of TRR379
|
||||||
|
|
||||||
with the aim to enable reproducible research from primary data to published
|
with the aim to enable reproducible research from primary data to published
|
||||||
|
|
|
||||||
|
|
@ -18,7 +18,7 @@ Contact the management team to request a dedicated calendar.
|
||||||
|
|
||||||
## Public calendars
|
## Public calendars
|
||||||
|
|
||||||
All public calenders are available via CalDAV URL and can be included in any
|
All public calendars are available via CalDAV URL and can be included in any
|
||||||
calendar solution, such as Google calendar.
|
calendar solution, such as Google calendar.
|
||||||
|
|
||||||
The URLs follow the pattern `https://cal.trr379.de/public/<name>`,
|
The URLs follow the pattern `https://cal.trr379.de/public/<name>`,
|
||||||
|
|
@ -26,7 +26,7 @@ where `<name>` is the calendar name, as stated in the list below. For the
|
||||||
events calendar this is `https://cal.trr379.de/public/events`.
|
events calendar this is `https://cal.trr379.de/public/events`.
|
||||||
|
|
||||||
For use with Google calendar, replace `https://` with `webcal://`. For example,
|
For use with Google calendar, replace `https://` with `webcal://`. For example,
|
||||||
the events calendar can be added to Google calender with the URL
|
the events calendar can be added to Google calendar with the URL
|
||||||
`webcal://cal.trr379.de/public/events`.
|
`webcal://cal.trr379.de/public/events`.
|
||||||
|
|
||||||
Available public calendars:
|
Available public calendars:
|
||||||
|
|
|
||||||
|
|
@ -4,6 +4,6 @@ weight: 70
|
||||||
---
|
---
|
||||||
|
|
||||||
https://nb-query.trr379.de hosts the central [NeuroBagel](https://neurobagel.org) query interface of the TRR379.
|
https://nb-query.trr379.de hosts the central [NeuroBagel](https://neurobagel.org) query interface of the TRR379.
|
||||||
With this solution, data availability can be queried dynamically, to faciliate data access requests.
|
With this solution, data availability can be queried dynamically, to facilitate data access requests.
|
||||||
|
|
||||||
NeuroBagel supports federated queries and is therefore ideally suited for the decentralized research data management approach of TRR379.
|
NeuroBagel supports federated queries and is therefore ideally suited for the decentralized research data management approach of TRR379.
|
||||||
|
|
|
||||||
|
|
@ -12,6 +12,6 @@ Many institutions provide their own [homeservers](https://matrix.org/docs/matrix
|
||||||
If not, anyone can create a free account at https://app.element.io/.
|
If not, anyone can create a free account at https://app.element.io/.
|
||||||
|
|
||||||
The TRR379 Matrix space is only for TRR379 consortium members.
|
The TRR379 Matrix space is only for TRR379 consortium members.
|
||||||
To get access, email [trr379@lists.fz-juelich.de](mailto:trr379@lists.fz-juelich.de?subject=TRR379%20Matrix%20access) with your Matrix acccount name included, or contact @mih:matrix.org directly on matrix.
|
To get access, email [trr379@lists.fz-juelich.de](mailto:trr379@lists.fz-juelich.de?subject=TRR379%20Matrix%20access) with your Matrix account name included, or contact @mih:matrix.org directly on matrix.
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -9,7 +9,7 @@ To assist with common issues you may encounter while working with the [knowledge
|
||||||
|
|
||||||
- It is **highly recommended** to use the knowledge pool's web UI with a desktop browser; the form offers rich contextual information that is hard to access on mobile devices.
|
- It is **highly recommended** to use the knowledge pool's web UI with a desktop browser; the form offers rich contextual information that is hard to access on mobile devices.
|
||||||
- Submitting records will only be possible using a [valid access token]({{% ref "/start/accounts/generate-token/" %}}); the token can be entered by navigating to "Settings" (gear icon) and then pasting your token in the "TOKENS" tab.
|
- Submitting records will only be possible using a [valid access token]({{% ref "/start/accounts/generate-token/" %}}); the token can be entered by navigating to "Settings" (gear icon) and then pasting your token in the "TOKENS" tab.
|
||||||
- When editing records, you cna hover your mouse pointer over field labels to display help text describing expected values and controlled vocabularies.
|
- When editing records, you can hover your mouse pointer over field labels to display help text describing expected values and controlled vocabularies.
|
||||||
- For some fields (e.g., journal name when filling out a Publication record), one or more choices have to be selected. The selector offers type-ahead search of choices. When searching:
|
- For some fields (e.g., journal name when filling out a Publication record), one or more choices have to be selected. The selector offers type-ahead search of choices. When searching:
|
||||||
- Try entering full names instead of abbreviations.
|
- Try entering full names instead of abbreviations.
|
||||||
- If an item is not found in one language, try searching its English version.
|
- If an item is not found in one language, try searching its English version.
|
||||||
|
|
@ -19,13 +19,13 @@ To assist with common issues you may encounter while working with the [knowledge
|
||||||
|
|
||||||
## Common errors
|
## Common errors
|
||||||
|
|
||||||
- **Authorization issues**: If the [access token]({{% ref "/start/accounts/generate-token/" %}}) you are providing is incorrect or does not have approriate permissions, the interface may desplay one or more of the following errors during metadata retrieval or submission:
|
- **Authorization issues**: If the [access token]({{% ref "/start/accounts/generate-token/" %}}) you are providing is incorrect or does not have appropriate permissions, the interface may display one or more of the following errors during metadata retrieval or submission:
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||

|

|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
- **Incomplete information**: If a metadata record is missing required information, you will not be able to save it. Required fields - highlighted with a red asterix - and an warning sign will alert you to this.
|
- **Incomplete information**: If a metadata record is missing required information, you will not be able to save it. Required fields - highlighted with a red asterisk - and an warning sign will alert you to this.
|
||||||
- **Internal errors**: If there is an unexpected error during metadata submission, the submission interface will report technical information. Please [contact the Q02 project team]({{% ref "/support/" %}}) and copy the entire error message.
|
- **Internal errors**: If there is an unexpected error during metadata submission, the submission interface will report technical information. Please [contact the Q02 project team]({{% ref "/support/" %}}) and copy the entire error message.
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue