Feature Req: usdz as a model in all browsers

[jeffscottward]

The lastest safari technology preview seems to now show the actual model in the browser when using Apple's proposed <model> tag with the flag enabled.

I suppose they are embedding their QuickLook engine inside Safari.

Autodesk has implemented a viewer themselves just recently without such a platform native dependency:
https://twitter.com/hybridherbst/status/1500266804471603205?s=21

Q: Can we embed this into model-viewer as a polyfill until Chrome and others natively support the <model> tag with .USDz support?

[elalish]

So far the <model> tag isn't really a replacement for anything but the barest use of <model-viewer>, and MV already has pretty universal browser/device support. If you really want to use .USDZ, go for it, but it's basically just glTF minus a clear spec and a standards body.

Is there anything in particular you're finding lacking in MV? I'm confident we can add features faster than any browser can.

[jeffscottward]

“Minus a clear spec and a standards body” is a bit reductive. Docs: https://graphics.pixar.com/usd/release/index.html Nvidia and most DCC’s are adopting USD as the authoring standard moving forward. They open sourced only a couple of years ago after it was pretty complete. Authors being late to the party should not be penalty to the end user. It has been proposed for discussion in w3c btw: immersive-web/proposals#69 Also, it’s a superset of GLTF where you can make self contained interactive scenes as one file as opposed to also needing a JS runtime and browser API’s. But beyond that, Apple is forcing it, so yea, we need a user experience that universally consistent, and that means supporting model viewing as USDz. *If a GLTF file alone had the same features as USDz, it would be a total non issue*. But simply put you can’t pull off Apple’s MTA transit demo interactivity with only a GLTF. Demo: https://transit.applepay.apple/new-york#callout-quicklook “MV already has pretty universal browser/device support.” It does not in this regard, hence the request, but not the fault of MV of course, a matter of browser implementation (and absolute bleeding edge tech), which given this abstraction layer, we want to account for. “particular you're finding lacking in MV?” Just .USDz model viewing across browsers but this is a super recent development. @PixelPartner might have some comments though. *Long story short*: I can’t ask 3D artists to create a whole interactive scene in two totally separate domains (especially when they can’t code) to cover both android and iOS. For static models, JIT conversion it’s fine. But the interactive layer puts a fork in the road. Unless you know of an authoring tool and I tell users to use that uniformly exports the same experience to both WebXR and USDz in one shot.

On Tue, Mar 22, 2022 at 5:26 PM Emmett Lalish ***@***.***> wrote: So far the <model> tag isn't really a replacement for anything but the barest use of <model-viewer>, and MV already has pretty universal browser/device support. If you really want to use .USDZ, go for it, but it's basically just glTF minus a clear spec and a standards body. Is there anything in particular you're finding lacking in MV? I'm confident we can add features faster than any browser can. — Reply to this email directly, view it on GitHub <#3298 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJUO7TVXWQ72EMTVM3PTU3VBI3J5ANCNFSM5RMD4ORQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

-- *Thanks,* *Jeff Ward* *Front-end Developer* *Tel: 516-551-8624* ***@***.*** ***@***.***>* ***@***.**** <https://twitter.com/#!/jeffscottward>

[elalish]

To be clear, USD is a great and popular format. It's only USDZ we have difficulty with, as it's a totally different beast. Write an exporter for it and you'll see. Still, as you say, everything on the bleeding edge is in flux. MV focuses more on the 3D side than the AR side, as that's where the majority of our users are finding value. Being web tech, we also tend to push the idea of interactions and such being defined in JS rather than in a file format, as we see that as a more flexible and sustainable long-term solution, but it's predicated on things like WebXR support becoming universal, which is anything but certain. And it's also web-first rather than native-first, which plenty of people and companies have differing opinions about.

Anyway, thank you for the explanations. We both want the same things (write once, run everywhere 3D and AR experiences). We're not putting our bets on USDZ to deliver, but we could easily be wrong. Only time will tell.

[PixelPartner]

To call MV "web-first" there should be a "next", but the community is depending on / hoping for WebXR and every web-dev to speak it fluently.
It's effectively a "web-only forever"!
Apple has proposed and is implementing model as its "next" step to ease the use of 3D on websites. Instead of "native first", I'd call Apple's roadmap "AR first".

I invite anyone interested in what's possible with an USD exporter (with focus on interactive schemes) to visit https://kreativekk.de/Swivel.html

Since 2 years RealityComposer awaits to activate command exchange between browser and 3D scene.
Trigger {string} from caller and
Action Send {string} to caller
can be used, but block any scene export - sigh.
I'm hoping WWDC22 to change that.

I have not had the time to implement it, but a converter from interactiveUSDZ to a standalone website with MV and WebXR interaction code is straightforward (given the current feature parity).

The other way round would be a manual reimplementation for every single use case.

[hybridherbst]

Just to chime in here as my tweet is quoted -

As much as I'd like to have support for USDZ everywhere, same as glTF, I have to agree with @elalish here that USD isn't well-defined and spec'ed. It's a "the code is the spec" situation, and from painful experience I know that pretty much every USD implementation is different - notably, Apple's own implementation that's used for QuickLook and presumably the model tag feature preview is broken in a myriad ways that are all not spec-conformant. This is the status for a couple years now, despite numerous bug reports. Here's a few of those bugs, funnily and kind of case-in-point using model-viewer to display working reference glTFs ;)

The Autodesk hydra delegate to render USDZ on the web (which usd-viewer is using) has the same issues: it currently only supports a very limited subset of USD features, notably things like skinned mesh renderers, audio, stages, vertex colors are all missing.

Given the above, a "model" shim, while interesting for technical considerations, would currently be pretty limited in practical use (with different behaviour in the Web, AR, and elsewhere). @jcowles since this is something we recently discussed.

If you want to give it a try / play with the idea, feel free to remix and adjust my https://usd-viewer.glitch.me/?file=
/ https://glitch.com/edit/#!/usd-viewer. Good luck :)