3D Tiles Roadmap

[pjcozzi]

New to 3D Tiles? See Introducing 3D Tiles.

This work is in the 3d-tiles branch.

In pseudo-priority order within each section:

Draft 1.0

• Cesium 3D Tiles implementation
  • Inspector, 3D Tiles Inspector #4846
  • Is the load order correct? The unit tests look OK; however, for example, the buildings on the right should load sooner: http://cesiumjs.org/WashingtonDC/
  • Memory issues?
    • Unloading issue, Unloading bug in 3d Tiles branch #4673
    • Empty tile memory usage, High memory consumption if tiles.json is big #3453
    • Crash with high memory usage, Scene Render Crashes with Large 3D Tiles #4549
  • glTF 2.0, Support for glTF 2.0 #4009
  • Merge request scheduler into master, Request Scheduler from 3d-tiles #3476
    • IE performance, 3D Tiles demo has terrible imagery performance #4166
    • Unable to have multiple CesiumWidets on the same page in 3d-tiles branch Unable to have multiple CesiumWidets on the same page in 3d-tiles branch #5226
  • Loose OBBs Oriented Bounding Box frustum check is too loose #4387
  • Translucency and artifacts
    • Combine translucent back/front commands into a single translucent command, however this will look worse when OIT is not supported.
    • Pick translucent features, see TODO in Cesium3DTileBatchTableResources.getVertexShaderCallback
    • Finish derived commands, especially Cesium3DTileBatchTableResources.getAddCommand, Derived commands #3806
    • Frustum artifacts 3D Tiles: Possible frustum artifact? #4381
  • Finish Styling GLSL-backend Shader styling considerations 3d-tiles#140
  • Styling - vector support for built-in functions 3D Tiles Styling - add vector support to the built-in functions #4865
• Doc and Examples
  • cesium.com hosted examples in the Sandcastle example
  • Tutorials
    • Using the Cesium API to load 3D Tiles and interact with individual models.
    • Declarative styling.
  • Copy 3D Tiles Sandcastle tests from the Specs/Data folder to the Apps/SampleData folder.
    • Or store test data in cesium.com and download on npm install or something similar?
• Any outstanding TODOs in the code
• Spec draft 1.0 issues
• Tools - 3d-tiles-tools
• drillPick point cloud - Calling drillPick() on a 3d-tiles point cloud without a limit hangs browser #4895

Post Draft 1.0

• Vector tile - instance geometry volumes, Vector Tiles #4665
• Vector tile - z-order, Vector Tiles #4665
• Vector tile and geometry tile - clipping planes
• If requested - classify with i3dm, Use b3dm tilesets for classification #6033
• Technique like Skip LODs #5128 for fuse tilesets/terrain at runtime
• Draco compression (then quantize at runtime), Draco mesh/point compression extension for glTF #5120
• Vector tile format. Vector Tile Roadmap 3d-tiles#25, Vector Data on Terrain #2172
  • Increase bounding volume by pixel size like BillboardCollection and PointCollection. See the Cesium3DTile constructor.
• Point cloud attenuation example using declarative styling (and vertex shader culling?)
• scene.pickPosition returns incorrect position scene.pickPosition returns incorrect position #4368
• Better error metric than SSE? maximumScreenSpaceError a bad metric for 3D Tiles #4043
• Reuse Primitive Batch Table, Batch table #4328
• Add convenience functions to the Cesium API for querying features based on the batch table hierarchy - like getFeaturesByProperty(propertyName, value) and getFeaturesByClass(className)
• Cache Management, @lasalvavida
  • Refactor TileReplacementQueue to use DoublyLinkedList?
  • Example code for app that uses tileUnload and tileVisible events to re-style a tile that was unloaded, e.g., by tracking features that changed or higher-level context like how 3D Tiles styling works.

Later + Ongoing Performance

• 3D Tiles spec and Cesium implementation
  • Add credit to tileset.json, Add credit to tileset.json 3d-tiles#55
  • Download multiple children in a single request, Download multiple children in a single request 3d-tiles#9
  • More declarative styling, Declarative styling 3d-tiles#2
  • Custom terrain tile format (instead of b3dm). Initially support overlaying separate imagery layers?
  • Refinement parameters. Additional tile metadata to drive refinement 3d-tiles#15
  • Texture LOD for b3dm/i3dm
  • Cleanup: GLSL bit extract helpers, 3D Tiles - Point Cloud color compression #4315 (comment)
• Cesium 3D Tiles implementation
  • AO for 3D buildings (integrate gltf-pipeline)
  • tileNewlyVisible and tileNewlyNotVisible events
  • Performance: Best SSE in center
  • Performance: Add extra frustum planes to account for how wide the FOV is and that the viewer might not notice objects on the edge and in the distance (horizon views for sure, maybe not aerial), e.g, two planes with normals perpendicular to the view direction
    • Also, tell users to use a more narrow FOV
  • Performance: cull/LOD select with union of children bounding volumes instead of the tile's BV. Does the extra CPU overhead result in fewer tiles on average? Is it a performance win?, Culling / OBB optimizations #2861
  • Performance: for fast moving views (like flying from home view to Canary Wharf), increase the geometric error so few tiles are requested and then only shown briefly or are already out of view. Could track the viewer's velocity and perhaps heuristics for if a requested tile was immediately culled
  • Performance: optimize replacement refinement - instead of requesting all children when a tile needs to refine, request only the visible ones, and then allow refining a tile if all the required visible tiles are loaded (instead of all - including invisible - tiles). When the camera moves, and children that are not yet loaded are needed, either render the parent or render the available children and the parent with clipping, 3D Tiles - refine to visible children in replacement refinement #4287
    • Still need to clip the parent to the loaded children
  • Performance: skip levels when refining with replacement refinement
    • For additive replacement, load tiles deep in the tree first, then ancestors
  • Performance: optimize batch table memory usage, see PERFORMANCE_IDEA in Batched3DModel3DTileContent.js and Cesium3DTileBatchTableResources.js.
  • Evaluate LZ4m: A Fast Compression Algorithm for In-Memory Data
  • Performance: Turn off opaque textures for picking pass, e.g., in Miami for rollover picking.
  • Performance: exploit temporal coherence in sortChildrenByDistanceToCamera
    • Sort by pixel size instead of distance? What about large bounding volumes with sparse contents?
    • Is selectTiles better implemented with a priority queue instead of explicit sort?
  • Performance: nest spatial data structure to optimize additive refinement, Nest spatial data structure to optimize additive refinement 3d-tiles#11
  • Performance idea: prefetch children or siblings? For example, see Figure 3. Track stats over time to change algorithm or disable.
  • Performance: Reduce per-DrawCommand overhead
  • Performance: Fuse nodes at runtime to reduce the number of DrawCommands.
  • Performance: "to avoid jarring popping artifacts, it could be useful for a runtime to scale the viewerRequestVolume to be slightly larger, and then check the geometricError for prefetching (but not rendering until the viewer enters the unscaled volume).", Refinement based on proximity 3d-tiles#101
  • Performance: devicePixelRatio on mobile, Honor devicePixelRatio take 3 #3288
  • Performance: Omni-directional Relief Impostors (ORI), Massive models and upper LOD levels implementation #4908 and https://groups.google.com/forum/#!topic/cesium-dev/tCCooBxpZFU%5B101-125%5D
  • Point clouds: hidden point removal, anisotropic filling, and shape depiction, and compute normals for lighting. Also edge-preserving blur in Game Engine Gems 3 and CIS 565.
  • Labels on buildings
  • Occlusion culling. Drive refinement with VMSSE?
  • Morph between LODs
  • Support 2D and Columbus view. 3D Tiles in 2D/CV #4884
  • Make PointGeometry and PointAppearance public. Added PointGeometry and PointAppearance #3203
  • Job scheduler, Unified time slicing tasks #2655
  • Memory-efficient (bitfield) per-building cast/receive shadows if needed
  • More Dynamic SSE 3D Tiles - Dynamic Screen Space Error #4307
    • Increase SSE as the view gets more horizon
    • Load tiles up to a max distance first, then load tiles in the distant if the camera doesn't move
    • Fog to help visual quality?
  • Cache Management
    • When skip LOD is used, remove non-leaf tiles from the cache if they are touched by not rendered
    • Research strategies for proactive cache trimming, e.g., number of seconds/frame a tile was not selected, how far out of view a tile is, etc.
      • Potentially track statistics to see if the cache is trashing, and then adjust the replacement algorithm.
    • Is it possible to come up with a better default for maximumNumberOfLoadedTiles since the use cases are so diverse?
    • Unload sub-trees from tiles with tileset.json content when all their content is unloaded
    • Vertex/texture cache across tiles
    • Provide more precise cache size than number of tiles? Count composite tiles as more? Include geometry/texture size with each tile? Perhaps number of tiles is the easiest thing to understand.
  • POP buffer for progressive replacement refinement
• Review "Interactive inspection of complex multi-object industrial assemblies", Massive models and upper LOD levels implementation #4908

Older

• gzip tiles? Should tiles always be gzipped? 3d-tiles#61, @pjcozzi
  • 404 for missing compressed tile, Send 404 error in case of compressed file not exists #3508, @pjcozzi
• Finish/optimize i3dm tile format. Instanced 3D Model updates 3d-tiles#33. @lasalvavida
• Finish point clouds tile format. Finish Point Cloud Tile spec 3d-tiles#22.
  • Compression notes: quantization; Morton curve to minimize deltas?; compress color to 11-bits like in Page 3? @lasalvavida
  • Rendering notes: point size attenuation, including vertex shader culling; styling, Add option to Cesium3DTileset to set custom appearance #3639
• Evaluate potential Batch Table changes, Batch Table changes 3d-tiles#32, @pjcozzi
• Instanced models are not being cached correctly between different ModelInstanceCollections, Instanced models are not being cached correctly between different ModelInstanceCollections #4144
• 3D Tiles cache not cleaning up fully 3D Tiles cache not cleaning up fully #4147
• tiles invisible and cut, 3D tiles - tiles invisible and cut #4333
• Do not set Model.incrementallyLoadTextures to true (default)
• b3dm with external textures reference error b3dm with external textures reference error #3892
• Fix test failures when running with Karma (makes Travis CI fail): npm run test -- --exclude WebGL --browsers Electron --failTaskOnError --suppressPassed
• Implement WEB3D_quantized_attributes. @lasalvavida
• Remove workaround2657 in Cesium3DTile.js, Primitive exception for empty rectangle #2657. @lasalvavida
• Fix replacement refinement when external tileset is loaded, Fix replacement refinement when external tileset is loaded #3517, @lilleyse
• Reorganize load/replacement (like terrain) to support shadows, @bagnell
• Add fog, @bagnell
• TileOrientedBoundingBox debug volume bug, 3D Tiles - TileOrientedBoundingBox debug volume should be 2x2x2 #4581
• debug show view frustum, Shadows #3856 (comment), @lasalvavida
• Fix mouse in Sandcastle example, 3D Tiles Sandcastle example has mouse issues in IE 11 #3614
• Expose bounding volume, Cesium3DTileSet should expose overall bounding volume #3138
• Support replacement color in addition to highlight color? See Cesium3DTileBatchTableResources.getFragmentShaderCallback.

[e-andersson]

Does "Cache management and replacement" include handling of which Cesium3DTilesets will be loaded or is it aimed at the lower-level tile handling? We're quite interested in tileset loading/unloading since we'll potentially have a large number of tilesets available to the viewer, causing a lot of initial tiles.json loading and repeating iteration over primitives.

[pjcozzi]

Does "Cache management and replacement" include handling of which Cesium3DTilesets will be loaded or is it aimed at the lower-level tile handling?

It means lower-level tile handling, but thanks for reminding me about which Cesium3DTileset to load. I added that to the roadmap. :)

If you are interested in working on either, I'm happy to discuss in more detail.

[pjcozzi]

Stick with the Cesium3DTile prefix for names?

After doing a bunch of reference doc, I don't have a strong preference, but I am find with this, e.g.,

new Cesium.Cesium3DTileset

Compared to

new Cesium.Tileset3D

new Cesium.Tiles3D

new Cesium.ThreeDTileset

Cesium.Tileset3D would also be OK. @lilleyse what do you think?

[pjcozzi]

@TomPed I added a section above for you for when you start on 3D Tiles.

[TomPed]

Rename Cesium3DTile#readyPromise to Cesium3DTile#contentReadyPromise

@pjcozzi does this mean the field readyPromise in the file Cesium3DTile.js?

[pjcozzi]

@TomPed yes.

[TomPed]

Likewise, replace isReplacementRefinable and isContentUnloaded with readonly properties. Complete doc.

@pjcozzi I can't seem to find isReplacementRefinable in the project.

[lilleyse]

Ah, don't worry about that we removed that. I'll update the task.

[TomPed]

3DTileContentProvider

Should this be Cesium3DTileContentProvider?

[lilleyse]

Yeah. The asterisks are just saying all things named like that.

[pjcozzi]

Potentially useful reading on V8 GC: https://medium.com/@_lrlna/garbage-collection-in-v8-an-illustrated-guide-d24a952ee3b8

[rory-ye-nv]

what's the status for vector tiles? it's no update or example, and no spec found in 3D tile 1.0