Skip to content

Conversation

@pavelkumbrasev
Copy link

When threads execute get_task or steal_task they can omit unsuitable tasks (different isolation or affinity). The omitted tasks are virtually extracted from task pool, that makes them temporary invisible for other threads. It might lead to situation when another thread marks the arena EMPTY (as a result of is_out_of_work()), while these tasks are still in the arena. If tasks were omitted we need to notify the arena like new work was added (calling advertise_new_work()).

The proposed idea is that omit tasks in such way that other threads can not observe it. It will prevent unnecessary arena state transition and premature threads leaving.

Therefore, the patch introduce shadow tail and head which move after real task extracted(do not participate in tasks omitting mechanism).

Signed-off-by: pavelkumbrasev [email protected]

@pavelkumbrasev
Copy link
Author

@aleksei-fedotov @anton-potapov @alexey-katranov, could you please review this patch?

@aleksei-fedotov
Copy link
Contributor

@aleksei-fedotov @anton-potapov @alexey-katranov, could you please review this patch?

Yep. Working on it!

//! Index of the element following the last ready task in the deque.
/** Modified by the owner thread. **/
std::atomic<std::size_t> tail;
std::atomic<std::size_t> shadow_tail;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a comment what shadow_tail is for and how different it is from the tail.

//! Index of the first ready task in the deque.
/** Modified by thieves, and by the owner during compaction/reallocation **/
std::atomic<std::size_t> head;
std::atomic<std::size_t> shadow_head;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a comment what shadow_head is for and how different it is from the head.

pavelkumbrasev added 2 commits July 11, 2025 18:13
@akukanov akukanov force-pushed the dev/pavelkumbrasev/reworked_missed_wake_up_avoidance branch from e3f2165 to 40b4e23 Compare July 11, 2025 18:34
@akukanov
Copy link
Contributor

akukanov commented Jul 11, 2025

Rebased on top of the current code base. However, I think the patch needs to be revisited first. Two tests failing on ARM likely mean that "shadowing" the head and the tail may require some extra synchronization.

The following tests FAILED:
5 - test_collaborative_call_once (Subprocess aborted)
66 - test_task (Subprocess aborted)

Update: apparently, the patch does not even achieve its goal, because reset_task_pool_and_leave, which also resets the shadow state, can be used in the middle of get_task while omitted tasks still remain in the pool (yes, this sounds strange - but the actual bounds are stored in get_task as H0 and T0, and restored later).

@akukanov
Copy link
Contributor

Reimplemented in a different way in #1779; closing this PR.

@akukanov akukanov closed this Jul 17, 2025
@akukanov akukanov deleted the dev/pavelkumbrasev/reworked_missed_wake_up_avoidance branch July 17, 2025 10:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants