Sound alignment is off

[Illiander42]

When trying to do tick-aligned single-tick sounds and splicing them together they don't line up properly and sound awful.

Blueprint string and mod in attached zip. The blueprint alternates between playing the complete sound in a single file, and the tick-length parts in a run to demonstrate the issue. If you want to try my generator script (also included) you'll need to independently download the soundfont from here as it makes the zip too large for the forums.

I'm doing this as a workaround due to the speaker not having good control over sound playback. I'm after tick-exact control over the sound stopping, and also sound loops.

The same issue would appear if I used longer samples. Using single-tick samples just makes it the most obvious.

[Rseding91]

I'll hand this over to the sound guy but I doubt this will ever work as you want. Sounds just aren't designed with that amount of control in mind in Factorio.

[Qon]

What does "not line up properly" mean, more precisely? Off by a few ticks sometimes, or off by microseconds?

For the latter, it's not really possible to solve this in (using, not writing) a game engine with sound clips. You can't make each tick exactly the same length.

If you had direct access to the sound buffer, high resolution timers and guaranteed cpu time on some interval then you could generate tones of varying length to account for the variability. This is very far from what combinators/lua/c++ in game code (as in not engine code) are able to process though.

But a sound library would probably handle looping playback and queued operations for switching playback file or stopping or changing volume in close-enough to real time (per tick). That could be exposed to Lua api and speaker entity.

[Illiander42]

"Not line up properly" means you can hear a very clear buzz when playing the sequence of tick-long sounds.

As for what I'm trying to work around: Factorio speakers currently don't stop playing when you stop the control signal, you can't adjust the volume of the sound currently being played, and the sound loop always goes to the start of the sound file.

If I had those options, I wouldn't need to try workarounds like "have a seperate sound file for each tick."

---

This is starting to sound more like a feature request the more I think about how impossible it is to work around. Thought I was being clever with the "well I'll just play a whole bunch of short sounds in sequence so I can control playback myself."

[Donion]

Unfortunately, what you're trying to do here is not possible with what's available.
You have the game's update interval fighting against the SDL's audio thread and the way how audio data are sent.
For seamless transition you would need queue samples and let the audio backend do it for you (this isn't available).

This indeed sound more like a feature request for the programmable speaker (I can't promise anything since we have an expansion to finish but I'm open to expanding programmable speaker at some point).

Side note: the idea of playing a bunch of short samples in sequence is clever, it just opens a whole new can of worms .

[xykite]

One way you're making things hard for yourself is that you're fighting physics: there is essentially no chance of this working if you use a lossy codec like vorbis, due to the artefacts you're introducing in the frequency domain. Getting a real codec / player to support gapless playback with one join is hard enough; supporting 60 joins / sec is not sensible.

This is at least something you can fix yourself: just start again but use .wav files instead of .ogg files. Also make sure you resample to a frame rate which is a multiple of 60 (oggdec gives a sample rate of 14000, which isn't).

Even then, I'm not optimistic - but good luck.

[Illiander42]

Thanks for the fast response and everything. Guess I'm putting this away until after the expansion

Would you like me to write up a proper feature request in the Suggestions forum?

[Donion]

Would you like me to write up a proper feature request in the Suggestions forum?

That would be great.

[boskid]

Given that programmable speakers were never designed with this use case in mind and since there is no sample queue and at the same time the update is not guaranteed to run at 60 updates per second in case of large maps or slow machines, i am going to call this is Not a bug.

[Illiander42]

Would you like me to write up a proper feature request in the Suggestions forum?

That would be great.

Detailed feature request in the pile

[mmmPI]

@mmmPI: There was a bit of discussion about why Factorio cannot do proper sound editing here.

I was doing some test on that note with 44.100 hz wav because it can divide by 60 , i recreated the sound in your test in audacity by picking up only 1 wavelengh and then copy pasting it with the proper number of points sampling points on a 44.100 hz workspace , to avoid the clipping. It worked , i could recreate a continuous sound from the single wavelength, but then i think such sound would have a duration of a non-integer number of ticks if you want to isolate the single wave. Or it would have to be a sound without proper enveloppe and the clipping will be heard. I'm not sure what you were trying to do, i wish i could help. This is the recreated sound from a single wavelengh taken from the 16th tick of test folder linked in the other thread that start and end in a neutral point :

note-16.rar

(24.94 KiB) Downloaded 14 times

The problem with sound that have a lengh of 1 tick and clipping is about physic to me. The clipping is heard when the curve representing the sound is at some moment having a discontinuity.

1 wave.jpg (117 KiB) Viewed 5505 times

upper part is the attached .rar
bottom part the original 1 tick sample

This single wave length that i duplicated has a certain duration, and the sound for it not to clip without enveloppe, it would need to be cut "perfectly" at a place where it is 0 like the seleted area. If the sound start or end when the wave is not in the middle the initial or even finishing clap can be heard. This is the case in the sample number 16 from your test.
As such, if such sample is played alone it will make some noise. Having the previous one queued would remove the clapping i suppose. But an isolated segment 16 would always clap unless if cut with neutral curve at start and end. ( good thing your note is not made of frequency of opposite phase on stereo track , this is a nightmare to sample without clipping ).

If using 1 tick long sound, except for rare frequency that fit into a tick perfectly, like 60 hz or 120 hz or 360 hz, each individual segment could end up looping seemlessly like a Gif, but otherwise 82 HZ for example, means that the wavelengh is not going to fit an integer number of time in a tick measured precisely at 1/60 of second or 60 HZ. In such case i don't know how programs deal with sounds. Namely how much time in microsecond between 2 sound.

This is related i think to the sampling rate again, the number of points that are used to represent the curve for 1 tick as visible in the picture, the .Ogg at 14.000 hz but for the wav you need to zoom in because there are a more points :

clipping.jpg (91.54 KiB) Viewed 5505 times

In audacity the spacing between the 2 left and right parts is automatically choosen to adapt the sampling rate. But for the lower part, the lower sampling rate could require special treatment , as 60 hz for the game framerate and 14000 hz are not working the same as 60 and 44.100 it means somehow the 2 sound being played following each other need to have a little offset to reproduce that difference :

clipping2.jpg (95.07 KiB) Viewed 5505 times

The bottom part represent i suppose what is being heard in the game currently when looping such segment. I don't know about the timing, the spacing between the left and right part of the "note 16" at the bottom. But i know the annoying clipping sound can come from either left and right side of the curve not aligning which i feel is bound to happen if the sample is 1 tick long but you want different frequency on it, and the clipping not being done in a place where it's neutral when starting a sound, this will cause a clap.

I only see problems when i look at things this way, but when playing somehow it matters less. Some of the "problem in theory" do not materialize, if the gap is tiny, and there are other sounds for example. I'm interested in the solution to those questions , it makes me curious

[coppercoil]

it just opens a whole new can of worms .

Not a real problem for Factorio engineers
CPU - done
Video clips - done
3D game - done
DAW - to do

[Donion]

ha! you're not wrong

[kpreid]

One way to work within the limitations is to perform granular synthesis: instead of playing 1-tick sounds that are intended to splice perfectly, play longer sounds that are designed to get overlapped randomly and blend together to the final result.

[mmmPI]

One way to work within the limitations is to perform granular synthesis: instead of playing 1-tick sounds that are intended to splice perfectly, play longer sounds that are designed to get overlapped randomly and blend together to the final result.

It's difficult to obtain a precise result in game because the sound can only be offset from one another by a full 1 tick. It is possible to play different sound at different volume to obtain "new" sound, like a open hi hat can ressemble a closed hi hat and a shaker and a crash when the later 2 volume is very low so that it sound like they are the resonance of the first one since they last longer their sustain is still heard when the hi hat has faded out.

With the mod adding speakers that play the note until the signal is removed, it can happen that there are some diphasing, that end up pleasing like https://en.wikipedia.org/wiki/Phase_music but to me although granular synthesis concepts are interesting to experiment with, i think the tools are still limited in factorio but i'd be curious to learn more about it.