browsing the sqlite dbs to find info about unprcessed crawler and inference jobs

[sprior]

I've got Karakeep v0.27.1 running via Docker Compose and it's been working and is very useful. When I go to the background jobs page it says there are 6 unprocessed crawler jobs and 55 unprocessed inference jobs and I'm trying to figure out what they are. I've got a smart list which shows untagged bookmarks and there are only 2 of those and they're text notes so that makes sense.

To try to figure out what's going on I copied the db.db and queue.db files out of the container and I'm browsing the databases with a DB Browser for SQLite, but so far I don't see where those unprocessed items would be represented in the database. When I look at queue.db I figured the tasks table would be the obvious choice, but there are only 11 records in that table and all the numRunsLeft's are 0. I see drizzle_migrations with 62 entries so that's sort of in the ballpark.

Can anyone give me some guidance where to find these unprocessed items to at least understand them and possibly clean them out? These have been present for weeks/months so it doesn't seem to be a just wait thing. I believe I've tried to recrawl failed links only, regenrate AI tags for failed bookmarks only, and regenerate AI summaries for failed bookmarks only and none of them have cleared out the unprocessed items.

[MohamedBassem]

The pending inference jobs is probably this bug: #1605

You can find the ones that are pending inference you can do:

select * from bookmarks where taggingStatus='pending' or summarizationStatus='pending';

and to find the pending crawling, you can do:

select * from bookmarkLinks where crawlStatus='pending'

[sprior]

OK thanks. I used those queries to figure out what the problem bookmarks were and then I cleaned them up by deleting and recreating most of them. It did give me some insights - not all of these were related to the v25 upgrade, some were more recent.
It appears that you determined that some PDF and image files could not be crawled for summaries and disabled the summarize button so they can't be given summarization pending status. I think that notes entered directory without a URL also cannot be summarized and should be added to the skip list. It also suggests at least a partial automated fix - iterate through all the bookmarks with a pending status and determine if the file/bookmark type is impossible to summarize like you do for new bookmarks and fix the status on those in place.