Skip to content

Conversation

@bensonr
Copy link
Contributor

@bensonr bensonr commented Aug 11, 2023

Changes behavior of reads for data that is not domain specific. The approach is now to have a single MPI-rank read any data and broadcast it to the other members of a specified pelist. This is to help reduce impacts on a distributed filesystem for large core counts and small IOs.

Fixes # (will be added after testing)

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Please also note
any relevant details for your test configuration (e.g. compiler, OS). Include
enough information so someone can reproduce your tests.

Checklist:

Please check all whether they apply or not

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • Any dependent changes have been merged and published in downstream modules

@laurenchilutti
Copy link
Contributor

@bensonr do we still need to keep this PR open, or can we close?

@bensonr
Copy link
Contributor Author

bensonr commented Jul 11, 2025

It is something we could still use, but I'll defer to the FV3 team as to whether they want this update.

@lharris4
Copy link
Contributor

I am interested in anything that might improve initialization time for our increasingly high-resolution runs. Currently it takes about 10--15 minutes to initialize X-SHiELD (27.6K cores, io_layout=4,4) when reading the native-grid prescribed aerosol files.

Do we have any testing results that this change should indeed make reads for large core counts more efficient?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants