reproducibility

[robbiemu]

The readme says:

The model was not fine-tuned on a specific voice. Hence, you will get different voices every time you run the model. You can keep speaker consistency by either adding an audio prompt, or fixing the seed.

So one would think that by fixing the seed we would get the same voice each time, but this is demonstrably false at least with cpu and mps. Even with an expanded seed setting for modern devices like:

def set_seed(seed: int):
    """Sets the random seed for reproducibility."""
    random.seed(seed)
    np.random.seed(seed)
    torch.manual_seed(seed)
    torch.use_deterministic_algorithms(True)
    if hasattr(torch.backends, "cuda") and torch.cuda.is_available():
        torch.Generator(device=torch.device("cuda")).manual_seed(seed)

        torch.cuda.manual_seed(seed)
        torch.cuda.manual_seed_all(seed)

        # Ensure deterministic behavior for cuDNN (if used)
        torch.backends.cudnn.deterministic = True
        torch.backends.cudnn.benchmark = False

        if os.getenv("CUBLAS_WORKSPACE_CONFIG") is None:
            os.environ["CUBLAS_WORKSPACE_CONFIG"] = ":4096:8"
    elif hasattr(torch.backends, "mps") and torch.backends.mps.is_available():
        torch.Generator(device=torch.device("mps")).manual_seed(seed)
        torch.mps.manual_seed(seed)
        # Enable deterministic behavior for MPS
        os.environ["PYTORCH_MPS_DETERMINISTIC"] = "1"
    else:
        torch.Generator(device=torch.device("cpu")).manual_seed(seed)

This does not work when run each time across multiple calls to generate() on the same instantiated Dia.from_pretrained (this is using the nari-labs/Dia-1.6B-0626 model).

specific use can be seen in https://github.com/robbiemu/bomdia in the src/components/audio_generator/tts.py file.

I realize that I can generate from a list (and I'm about to see if making that is an improvement) instead of separate calls to generate per "chunk" of the transcript to generate, but I was also logging the seed for reproducibility/resumability and that doesn't seem possible.

[robbiemu]

Yea I just switched locally to using the batch mode and unsampled voices still change within the batch.

The only edits I have made to the project are:

git diff main
diff --git a/dia/model.py b/dia/model.py
index 56a34ef..877de6f 100644
--- a/dia/model.py
+++ b/dia/model.py
@@ -116,7 +116,7 @@ class Dia:
         self.device = device if device is not None else _get_default_device()
         if isinstance(compute_dtype, str):
             compute_dtype = ComputeDtype(compute_dtype)
-        self.compute_dtype = compute_dtype.to_dtype()
+        self.compute_dtype = compute_dtype
         self.model: DiaModel = DiaModel(config, self.compute_dtype)
         self.dac_model = None
         self._compiled_step = None
@@ -204,7 +204,7 @@ class Dia:
 
         # Load model directly using DiaModel's from_pretrained which handles HF download
         try:
-            loaded_model = DiaModel.from_pretrained(model_name, compute_dtype=compute_dtype.to_dtype())
+            loaded_model = DiaModel.from_pretrained(model_name, compute_dtype=compute_dtype)
         except Exception as e:
             raise RuntimeError(f"Error loading model from Hugging Face Hub ({model_name})") from e
 
diff --git a/pyproject.toml b/pyproject.toml
index dd844dd..fc7ac04 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -3,21 +3,27 @@ name = "nari-tts"
 version = "0.1.0"
 description = "Dia - A text-to-speech model for dialogue generation"
 readme = "README.md"
-requires-python = ">=3.10"
+requires-python = "==3.10"
 license = {file = "LICENSE"}
 authors = [
     {name = "Nari Labs", email = "contact@narilabs.ai"}
 ]
 dependencies = [
     "descript-audio-codec>=1.0.0",
-    "gradio>=5.25.2",
-    "huggingface-hub>=0.30.2",
+    "gradio~=5.42.0",
+    "huggingface-hub~=0.34.4",
+    "pydantic~=2.11.7",
+    "safetensors~=0.6.2",
+    "soundfile~=0.13.1",
+    "torch~=2.8.0",
+    "torchaudio~=2.8.0",
     "numpy>=2.2.4",
-    "pydantic>=2.11.3",
-    "safetensors>=0.5.3",
-    "soundfile>=0.13.1",
-    "torch==2.6.0",
-    "torchaudio==2.6.0",
+    "librosa>=0.10.0",      # 0.10+ already supports 3.10
+    "numba>=0.56",          # brings llvmlite>=0.39
+]
+
+[project.optional-dependencies]
+cuda = [
     "triton==3.2.0 ; sys_platform == 'linux'",
     "triton-windows==3.2.0.post18 ; sys_platform == 'win32'",
 ]

so: changes needed for torch 2.8, and optional cuda rather than required. nothing that touches randomization.

[robbiemu]

I ran a couple of experiments:

1. generators per item in batch
   I changed model.py so that _sample_next_token accepted an array of generators that were created in generate from a seed parameter.

        # Create a deterministic generator for voice sampling once per call
        gens = None
        if voice_seed is not None:
            B = len(text) if isinstance(text, list) else 1
            gens = []
            device = str(self.device or _get_default_device())
            for _ in range(B):
                g = torch.Generator(device=device)
                g.manual_seed(int(voice_seed))
                gens.append(g)
...
    if generators is None:
        sampled_indices_BC = torch.multinomial(final_probs_BCxV, num_samples=1)
    else:
        sampled_rows = []
        for i, row in enumerate(final_probs_BCxV):
            g = generators[i % len(generators)]  # cycle if more channels than batch
            idx = torch.multinomial(row, num_samples=1, generator=g)
            sampled_rows.append(idx)
        sampled_indices_BC = torch.cat(sampled_rows, dim=0)

Now the random number is starting at the same initial state for all streams. But this does not work.

2. I tried setting the values deterministically at the start of each batch for K steps (ie current_idx), like:

    if step < 15:
        sampled_indices_BC = logits_BCxV.argmax(dim=-1, keepdim=True)
    else:
        sampled_indices_BC = torch.multinomial(final_probs_BCxV, 1)

But this did not work either.

[robbiemu]

generators per item in batch seems much more likely to work if also voice cloning. strangely, voice cloning alone does not (I thought that was not a problem but apparently it is)

[robbiemu]

also, be wary of "(chuckle)" -- so often that sounds like hot air rushing through a metal tube or something.