error when initializing the model

[sivannavis]

Hi! I'm just following the tutorial doing these:

import openl3
model = openl3.models.load_audio_embedding_model(
    input_repr="mel128",
    content_type='music',
    embedding_size=512,
)

And an error pops out:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/ext3/miniconda3/lib/python3.11/site-packages/openl3/models.py", line 134, in load_audio_embedding_model
    return load_audio_embedding_model_from_path(model_path, input_repr, embedding_size, frontend=frontend)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/ext3/miniconda3/lib/python3.11/site-packages/openl3/models.py", line 166, in load_audio_embedding_model_from_path
    m = AUDIO_MODELS[input_repr](include_frontend=frontend == 'kapre')
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/ext3/miniconda3/lib/python3.11/site-packages/openl3/models.py", line 413, in _construct_mel128_audio_network
    spec = __fix_kapre_spec(get_melspectrogram_layer)(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/ext3/miniconda3/lib/python3.11/site-packages/openl3/models.py", line 44, in get_spectrogram
    seq = func(*a, return_decibel=False, **kw)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/ext3/miniconda3/lib/python3.11/site-packages/kapre/composed.py", line 261, in get_melspectrogram_layer
    return Sequential(layers, name=name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/ext3/miniconda3/lib/python3.11/site-packages/keras/src/models/sequential.py", line 74, in __init__
    self._maybe_rebuild()
  File "/ext3/miniconda3/lib/python3.11/site-packages/keras/src/models/sequential.py", line 139, in _maybe_rebuild
    self.build(input_shape)
  File "/ext3/miniconda3/lib/python3.11/site-packages/keras/src/layers/layer.py", line 222, in build_wrapper
    original_build_method(*args, **kwargs)
  File "/ext3/miniconda3/lib/python3.11/site-packages/keras/src/models/sequential.py", line 180, in build
    x = layer(x)
        ^^^^^^^^
  File "/ext3/miniconda3/lib/python3.11/site-packages/keras/src/utils/traceback_utils.py", line 122, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/ext3/miniconda3/lib/python3.11/site-packages/optree/ops.py", line 594, in tree_map
    return treespec.unflatten(map(func, *flat_args))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Exception encountered when calling STFT.call().

Could not automatically infer the output shape / dtype of 'stft_3' (of type STFT). Either the `STFT.call()` method is incorrect, or you need to implement the `STFT.compute_output_spec() / compute_output_shape()` method. Error encountered:

Invalid dtype: complex64

Arguments received by STFT.call():
  • args=('<KerasTensor shape=(None, 1, 48000), dtype=float32, sparse=None, name=keras_tensor_7>',)
  • kwargs=<class 'inspect._empty'>

Any idea what I did wrong here? Thanks!

[vdng9338]

Same thing here! I followed the instructions in README.md to install OpenL3 on Ubuntu 22.04 (within a fresh Conda environment), then followed the first steps of the tutorial and got this error:

Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import openl3
2024-04-25 18:34:04.432317: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-04-25 18:34:04.432638: I external/local_tsl/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-04-25 18:34:04.434472: I external/local_tsl/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-04-25 18:34:04.461994: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI AVX512_BF16 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-04-25 18:34:04.867150: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
>>> import soundfile as sf
>>> audio, sr = sf.read("audiofile.wav")
>>> emb, ts = openl3.get_audio_embedding(audio, sr)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/victor/.local/lib/python3.10/site-packages/openl3/core.py", line 262, in get_audio_embedding
    model = load_audio_embedding_model(
  File "/home/victor/.local/lib/python3.10/site-packages/openl3/models.py", line 134, in load_audio_embedding_model
    return load_audio_embedding_model_from_path(model_path, input_repr, embedding_size, frontend=frontend)
  File "/home/victor/.local/lib/python3.10/site-packages/openl3/models.py", line 166, in load_audio_embedding_model_from_path
    m = AUDIO_MODELS[input_repr](include_frontend=frontend == 'kapre')
  File "/home/victor/.local/lib/python3.10/site-packages/openl3/models.py", line 518, in _construct_mel256_audio_network
    spec = __fix_kapre_spec(get_melspectrogram_layer)(
  File "/home/victor/.local/lib/python3.10/site-packages/openl3/models.py", line 44, in get_spectrogram
    seq = func(*a, return_decibel=False, **kw)
  File "/home/victor/.local/lib/python3.10/site-packages/kapre/composed.py", line 261, in get_melspectrogram_layer
    return Sequential(layers, name=name)
  File "/home/victor/.local/lib/python3.10/site-packages/keras/src/models/sequential.py", line 74, in __init__
    self._maybe_rebuild()
  File "/home/victor/.local/lib/python3.10/site-packages/keras/src/models/sequential.py", line 139, in _maybe_rebuild
    self.build(input_shape)
  File "/home/victor/.local/lib/python3.10/site-packages/keras/src/layers/layer.py", line 222, in build_wrapper
    original_build_method(*args, **kwargs)
  File "/home/victor/.local/lib/python3.10/site-packages/keras/src/models/sequential.py", line 180, in build
    x = layer(x)
  File "/home/victor/.local/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py", line 122, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/home/victor/.local/lib/python3.10/site-packages/optree/ops.py", line 594, in tree_map
    return treespec.unflatten(map(func, *flat_args))
RuntimeError: Exception encountered when calling STFT.call().

Could not automatically infer the output shape / dtype of 'stft' (of type STFT). Either the `STFT.call()` method is incorrect, or you need to implement the `STFT.compute_output_spec() / compute_output_shape()` method. Error encountered:

Invalid dtype: complex64

Arguments received by STFT.call():
  • args=('<KerasTensor shape=(None, 1, 48000), dtype=float32, sparse=None, name=keras_tensor_1>',)
  • kwargs=<class 'inspect._empty'>
>>> 

Using Python 3.8 solved the issue for me. Apparently, OpenL3 currently only supports Python 3.6 through 3.8, according to #94 (comment).