Specifications for vector tiles loading? (both .pbf and metadata.json)

[TacticalGator]

I cannot find any proper documentation anywhere regarding the specifications required for vector tiles loading (both .pbf and metadata.json).

I tried loading my own vector tiles & metadata that include a timestamp in ISO8601, but kepler.gl parses it as strings and it's not showing up in the filter.

Can someone point me to the right direction?

Edit 2:
I tried converting ISO8601 to unix timestamp and declare it as "number" on metadata.json. Kepler.gl parsed it as float and made it an available "field" in filter now.
"time", "date", "datetime" types don't seem to be available for vector times & metadata loading.
Can somebody help?

[igorDykhta]

Hi @TacticalGator,
It looks like this isn’t supported at the moment.
Tippecanoe (and the MVT spec) don’t have a native DATE/TIME type,
and type handling currently happens before we receive any data,
when we process the metadata.

I’ll add this as a feature request.

There’s a fallback in place right now: when proper fields are missing from the metadata, the inner logic tries to fetch (I think) the central tile based on the metadata bounds and then infers the types from that tile. Apparently this won't work in many cases.

[TacticalGator]

Hey @igorDykhta !
Thank you so much for taking a look into this.

There appear to be several implementations of vector tile "specs", most of which are wrapped in a binary format like protobuf. This got me real confused as to how I could implement one in order to fit into the Kepler.gl's ecosystem.

There are so many tool our there, but Tippecanoe being the sole vector tile example that I could find: https://github.com/keplergl/kepler.gl-data/tree/master/mvt_population

so I attempted to reconstruct it using a Python script.

In addition, only "float" data type is showing up and configurable in the filter. Everything else, boolean, int, string, etc are missing in the filter. I am not entirely sure if this is an error on my end though.

[igorDykhta]

@TacticalGator string filters with tilesets are definitely not supported.
int - hm, should be supported.

[igorDykhta]

https://cdn.unfolded.ai/indigo/hexify_v5/{z}/{x}/{y}.pbf
https://r2-public.protomaps.com/protomaps-sample-datasets/cb_2018_us_zcta510_500k.pmtiles
https://api.mapbox.com/v4/mapbox.mapbox-streets-v8/{z}/{x}/{y}.mvt?access_token=XXX
https://r2-public.protomaps.com/protomaps-sample-datasets/nz-buildings-v3.pmtiles
https://fsq-os-places-us-east-1.s3.us-east-1.amazonaws.com/release/vector-tiles/latest/fsq-os-places.pmtiles

[TacticalGator]

@igorDykhta Wow so quick!
Thank you so much for links! They are exactly what I needed to understand vector tiles in kepler.gl.
I will test it again with int

[igorDykhta]

Bool and int should definitely work. Seems like a bug.

[igorDykhta]

If you can generate some interesting examples, I can add them to the docs.

[TacticalGator]

Bool and int should definitely work. Seems like a bug.

@igorDykhta So the metadata for the first example link that you shared, has both integer(croptype) and float(season_means) attributes as "number" as well as a boolean(`till_class) attribute.

On the version 3.2.0, integer(croptype) is showing up as float while boolean is correctly shown.

If you can generate some interesting examples, I can add them to the docs.

I got some vector tiles that I've been testing locally. Not entirely sure how I could provide you some examples.

[TacticalGator]

Hi @TacticalGator, It looks like this isn’t supported at the moment. Tippecanoe (and the MVT spec) don’t have a native DATE/TIME type, and type handling currently happens before we receive any data, when we process the metadata.

I’ll add this as a feature request.

There’s a fallback in place right now: when proper fields are missing from the metadata, the inner logic tries to fetch (I think) the central tile based on the metadata bounds and then infers the types from that tile. Apparently this won't work in many cases.

Hello @igorDykhta
Any progress on this feature?

[igorDykhta]

@TacticalGator
Not really. What do you think is the better way to handle this? Add non-standard type hints in the metadata, or add an optional UI to define change types?

[TacticalGator]

@igorDykhta
Adding non-standard type hints is not really practiced with MVP and would become kepler.gl-specific if implemented.
I think that the latter is much more flexible and would give users more control.

Besides, most temporal data will come in either ISO8601, RFC3389, or the Unix timestamp format.
ISO8601 and RFC3389 seem pretty straight forward have Kepler.gl to "detect", whereas the Unix timestamp format might be challenging it would need range checks or context/heuristics, etc.

[igorDykhta]

Well, it won't be an automatic detection; instead, the user will need to provide additional information about the types, as we don't have sample data to infer the types from.

[TacticalGator]

@igorDykhta
Sounds good. I really wish that I had enough React and Redux knowledge to hop in.

There is also another feature idea that I had in mind. A unified time slider that users can add data sources to. I've been trying to hack my way around using LLMs but not much avail.

Please let me know once you implement this. I would love to test it out.