weight damage in .keras format

[innat]

Packages

• Keras-nightly

Describe

• Saved model in .keras format
• Reload it to check similarity but weight missmatch.

Gist

• colab

Others

Potential similar issue. #19019

[SuryanarayanaY]

Hi @innat ,

I have replicated the reported behaviour with tensorflow and torch backends also. Attaching gist-tf and gist-torch.

Note: I tested with explicit serialization/deserialization for custom objects and still behaviour is same. Need to check the reason for this behaviour.

[innat]

@SuryanarayanaY Thank you for the response. One of the suspected area in the code is mention below (possibly). Coz, if I save the Attention module itself to .keras format, it works (as far as I remember).

    def build(self, input_shape):
        self.safe_window_size = get_window_size(
            input_shape[1:-1], self.window_size
        )
        self.attention_layer = Attention(
            dim=self.dim,
            window_size=self.safe_window_size,
            num_heads=self.num_heads,
            qkv_bias=self.qkv_bias,
            qk_scale=self.qk_scale,
            attn_drop_rate=self.attn_drop_rate,
            proj_drop_rate=self.proj_drop_rate
        )

This is what I stated in this old ticket. tbh, the new saving strategy in keras 3 is bit confusing than before.

Let me know if I need to add any information.

[innat]

Any update? cc. @nkovela1