Menu Content/Inhalt
Home Entwicklerblog
  • English
  • German
  • Japanese


feed-image RSS 2.0

Willkommen im fre:ac-Entwicklerblog. Ich werde hier Statusupdates und andere Informationen zur fre:ac-Entwicklung posten.

fre:ac development status update 04/2017 Drucken E-Mail
Geschrieben von: Robert   
Mittwoch, den 05. April 2017 um 12:30 Uhr

Here's an update on fre:ac development in the past month. Development in April was very active with work on many different areas and I can finally talk about some upcoming features now.

More bug fixes

I talked a lot about bug fixes in the previous report and have some more of them for this issue, too:

  • Haiku OS: fre:ac running slowly when started with arguments
    This one was tricky and riddled me for quite a while. On Haiku OS, and only there, fre:ac would run very slowly - to a point where it would be unusable - when started with arguments on the command line. Start it without any arguments and it would run at normal speed.

    At first, I added some timing code and it showed that when running ./freac abc from the command line, many operations, like loading codecs and extensions, took much longer than when simply running ./freac. Really weird as fre:ac doesn't do anything with these arguments, but simply ignores them.

    I left this untouched for a while and came back after a few days. Fortunately, I quickly found out what was going on then: fre:ac tries to find the directory it is executed from in order to locate its resource files (language files, images, extensions etc.). This part is system specific and on Haiku the original implementation parsed the output of the ps command in order to get that path. This would normally yield something like /boot/common/non-packaged/bin/freac which could be interpreted as a filename to get the path. Unfortunately, when run with arguments, the arguments would be included in the ps output to give something like /boot/common/non-packaged/bin/freac abc which could no longer be interpreted as a filename. The routine would thus return a null path and would try to find the correct execution path again when invoked the next time, running and parsing ps over and over just to get the same null result again and again and causing a massive slowdown of the application.

    The solution was to query the path using the designated Haiku API, asking the global be_app object for it.

  • Windows: fre:ac hung when trying to stop paused playback
    This one also was quite tricky to fix: Due to a bug in fre:ac's DirectSound output component, it hung when trying to stop a previously paused playback. I.e., select a track, hit the play button, hit the pause button and finally hit the stop playback button and fre:ac would hang.

    The reason for this was the DirectSound output component reporting that samples could be written even if the buffers were already filled. It would then block when fre:ac actually tried to write the samples, waiting for the buffers to empty enough so they could take the new samples. When paused, however, the write function would wait and block indefinitely as the buffers would never be emptied.

  • OpenBSD: fre:ac hung because of timer bugs
    On OpenBSD, fre:ac uses a threads-based timer implementation for things like blinking cursors. This is necessary because OpenBSD does not support POSIX timers based on signals. The threads-based timer implementation, however, was not well tested and contained several synchronization bugs, causing fre:ac to hang after using the UI for a while.

    This was a little difficult to debug, unfortunately, because the GNU debugger available for OpenBSD was not very reliable. I ended up enabling threads based timers on Linux and debugging there. After a few hours of fixing synchronization issues, there now is a stable timer implementation for OpenBSD and other systems lacking POSIX timers (like GNU Hurd, yeah!).

Performance improvements

In addition to fixing bugs, I was working on further speeding-up conversions, finding several opportunities for improvement:

  • Waiting faster
    Sometimes a process has to wait for something else to be ready and there were several places in fre:ac's code with room for improvement in this regard.

    I had a look at all those places and improved many of them to use less CPU cycles while waiting and reduce overall waiting times. The former reduces fre:ac's CPU usage with some codecs and slightly improves conversion speed when using all CPU cores while the latter reduces the time taken for switching from one track to the next during conversions. Together, these improvements should provide nice speed-ups for conversions of multiple tracks, especially on processors without SMT/HT.

  • Improved duplicate filename check
    The check for duplicate filenames taking place before each conversion job can take quite some time when lots of files are involved. In a test setup, I had a conversion job with several thousand files spending 15 seconds for the check. By analyzing the involved routines with a performance profiler, I was able to identify where it spent most of the time and could reduce the time by a factor of 5. The aforementioned test setup now does the check in about 3 seconds instead of 15.
  • Faster single file conversions
    fre:ac uses a parallel conversion engine to speed up conversions of multiple files. When converting only a single file or when combining multiple input files into a single output file, however, only a single CPU core is used.

    I'm currently looking into ways to improve performance for such single file conversions, too, and so far, things are looking really good. I expect to be able to talk about this in more detail and present some performance numbers in next month's report.

Other improvements

Besides performance improvements, fre:ac also got some improvements in other areas:

  • macOS and Haiku: Improved CDDAFS handling
    macOS and Haiku expose audio tracks of inserted CDs as virtual .aiff and .wav files. This is great for quickly copying tracks to your computer without the need for a dedicated CD ripper. However, without special precautions, it gets in the way of the parallel conversion engine in fre:ac. fre:ac sees the tracks as files and tries to convert them in parallel - CD drives, however, are not very good at reading multiple tracks at the same time - they keep constantly repositioning the laser and the whole process gets really slow.

    The next snapshot release will notice when such virtual CD track files are added to the joblist and treat them like regular CD tracks, disabling parallel conversions for them.

  • Improved audio format support
    A user sent me some .m4a files that fre:ac was unable to open. They were obtained through an online YouTube to audio conversion service and turned out to be invalid, missing a table of contents section. The MP4v2 library used by fre:ac to access the audio content of .m4a files is unable to handle such files and thus, fre:ac was unable to open them. fre:ac also uses the avconv/ffmpeg utility to read some other file formats, though, and it turned out that this tool can read such invalid files. So what the next snapshot release will do is pass files on to avconv/ffmpeg if the dedicated .m4a decoder is unable to handle them. Reading the files that way is a bit slower, but at least you will be able to access and convert them.

    While at it, I also added support for reading .aea (ATRAC-1) files as per another user's request.

A DSP engine

The big feature of the next snapshot will be a DSP or digital signal processing engine. I talked about this already one year ago, but did not finish it back then.

This time, however, I'm positive that I can complete it for the next snapshot release. At first, it will provide sample rate conversion only. More filters will be added in following releases.

fre:ac 1.0.29

While all of the above applies to development towards a new snapshot release, I also implemented a significant change for the next stable release, fre:ac 1.0.29. That version will switch to using mpg123 instead of MAD as its MP3 decoder. Unlike MAD, the mpg123 library is under active development and handles some MP3 files much better than MAD. The mpg123 decoder has been the default MP3 decoder in fre:ac development snapshots for a while and should be very stable and reliable.

This closes this months report. Be sure to check for a new snapshot release around the end of May and come back for a new development update in early June.

fre:ac development status update 03/2017 Drucken E-Mail
Geschrieben von: Robert   
Dienstag, den 04. April 2017 um 20:42 Uhr

Hi folks, here's a new update on fre:ac development progress. Sorry for not giving an update last month - I was really busy preparing the March snapshot release. Thus, this update covers the past two months - February and March - and got rather lengthy.

fre:ac 1.0.28

fre:ac 1.0.28 was finally released in February with the FLAC codec updated to version 1.3.2 and an important fix for cover art handling. Previous versions of fre:ac had problems handling files with invalid tags that announced the presence of cover art, but contained no actual data. The 1.0.28 release fixes this.

20170317 snapshot release

After fre:ac 1.0.28 was out, I concentrated on finishing the snapshot release. Lots of improvements, such as title info extraction from file names,  per-folder playlists/cuesheets or reading embedded cuesheets, have been mentioned in previous posts already. Here are some more that I didn't cover here yet:

  • Support for album artist
    Likely the most important addition is support for the album artist tag field. fre:ac now reads and writes this value and adds an edit field for it in the tag editor. In addition, the new snapshot introduces an <albumartist> placeholder for the filename pattern and supports an optional joblist column for the album artist.
  • freaccmd improvements
    The freaccmd command line utility also got some improvements in the March snapshot. It now supports all encoders available in the GUI version and recognizes a new option to select from different available configurations.
  • HTTPS support
    The smooth Class Library which forms the base for fre:ac now uses libcurl for HTTP transfers and enables support of the HTTPS protocol. While there are not many places in fre:ac where that matters at the moment, it will become more important in future versions when the video downloader extension will be made available again.

At the moment - besides other things - , I'm working on creating packages for the Haiku operating system (mentioned here in December). I already made myself familiar with Haiku's package management system (which really is one of the best I've ever come across), but still have some issues to fix before I can provide official fre:ac packages.

The bug hunting season is open

After a release usually is bug hunting time - even more so with two releases from different branches, so in the past three weeks I was mostly fixing issues reported by users:

  • Non thread safe LAME decoder
    A user running fre:ac on Linux reported garbled output when converting MP3 files in multi core mode. This turned out to be an issue with the LAME MP3 decoder not being thread safe - which unfortunately is not mentioned in the LAME docs and thus was not taken into account by fre:ac. The next snapshot will fix that by disabling parallel mode for the LAME decoder. Fortunately, the MAD or mpg123 decoders, which do not have such issues, are used on most systems. Also note that this only affects the LAME decoder, not the encoder which is perfectly thread safe.
  • Problems with CIFS/SMB shares
    The current version has problems opening files and folders on mounted CIFS/SMB shares. This turned out to be the result of the GNU C library, glibc, and the CIFS mounting utility not playing together nicely. When calling stat() on a file or folder name to see if it actually exists, glibc will internally call stat64() and CIFS will generate a 64 bit inode number which glibc in turn will respond to with an overflow error, because it cannot convert it into a 32 bit value for the original stat() call. The solution is to use stat64() from the beginning, which fre:ac will do starting with the next snapshot.
  • Access violations with the FDK AAC decoder
    When reading AAC files with the FDK AAC decoder available in some Linux distributions, the decoder can generate an access violation by reading a few bytes past the supplied buffer containing the AAC sample. This will crash the application if the buffer ends near a memory page boundary and the next page is protected because it belongs to another process. The chance for this to happen seems to be quite low, however, as I got crashes only approximately once every 4000 files. The next snapshot will fix this by allocating slightly larger buffers for the FDK decoder. In addition, the decoder's behaviour will be documented in the next release of the FDK library.
  • Minor bug fixes
    In addition to the above, several minor bugs have been fixed. These are things like graphical glitches when displaying the progress bar or toggling the title info area. Too many to mention them all here.

This closes this months status update. Keep checking this blog section for the next update with an outlook on changes sheduled for the next snapshot.

fre:ac development status update 01/2017 Drucken E-Mail
Geschrieben von: Robert   
Freitag, den 03. Februar 2017 um 22:34 Uhr

Here's the fre:ac development status update for January 2017. It's already February now, so let's not lose any more time and go right ahead.

Work on the next snapshot

Since the last update, a lot of changes for inclusion with the next snapshot have been implemented. These are the most important ones:

  • Advanced info extraction from file names
    When no tag information is available for a file loaded into the joblist (as often happens with .wav files), current versions of fre:ac take the file name and treat the first part before a " - " delimiter as the artist and the last part as the title. So, for example, in Ed Sheeran - Shape of You.wav the artist and title will be correctly recognized. With more advanced naming schemes, fre:ac can easily get confused, though.

    The new algorithm knows and recognizes lots of different popular naming schemes. For example, it now takes the folder name into account to recognize patterns like <artist> - <album>\<track> - <title> and it looks for numbers in the file name and treats them as track numbers.

  • Easier format selection for codecs supporting multiple formats
    Selecting format when starting a conversionSome encoder components like Core Audio or SndFile support different output formats. For example, SndFile can create .wav or .aiff files. Starting with the next snapshot, fre:ac will allow you to select the desired output format directly from the Start encoding drop down menu. It's no longer necessary to go to the configuration dialog to change the output format.
  • Create one playlist / cue sheet per folder
    When converting multiple albums in one conversion run, current versions of fre:ac create a single playlist or cue sheet file with all the tracks of all albums, which is probably not what you want in most cases.

    Starting with the next snapshot, there will be an option (enabled by default) to create separate playlist and cue sheet files per output folder. So, if you use a pattern like the default <artist> - <album>\<artist> - <album> - <track> - <title>, you will get separate playlist and cue sheet files for each album.

  • Faster editing of tracks with cover art
    In the current snapshot version, performance can get really slow when editing title information of tracks with cover art. This will be fixed in the next snapshot thanks to fre:ac user Shri, who reported this behaviour as a bug on SourceForge.
  • Open the output folder from the main window
    Next to the name of the output folder at the bottom of the main window now is a button to open that folder in your systems file explorer.
  • Polishing for an upcoming beta release
    Progress indicator in macOS dockSeveral minor changes have been implemented to get fre:ac 1.1 closer to a beta release. For example, the macOS version now displays conversion progress in the dock, fre:ac's application name is no longer shown as unknown in the Gnome panel on Linux or FreeBSD and fre:ac recognizes configuration files of 1.0.x releases and keeps most settings when upgrading from such a version.

Some more changes are queued for the next snapshot, but are not quite ready yet, which is why there was no new snapshot release in January. I hope to be able to talk about those changes in next month's issue.

1.0.28 coming soon

I hoped to have a new stable release by now, but things turned out a little more complicated. A compilation bug was found in FLAC 1.3.2 and I decided to wait for a fixed release. However, probably due to lots of people submitting various fixes for FLAC 1.3.2, that have to be reviewed and integrated, a new version is not available yet.

Thus, I decided to just apply the patch for the compilation bug to FLAC 1.3.2 and make a fre:ac release with it. I expect to be able to push out fre:ac 1.0.28 in the next few days.

That's it for this update. Stay tuned for the 1.0.28 release and a new status update next month.

fre:ac development status update 12/2016 Drucken E-Mail
Geschrieben von: Robert   
Mittwoch, den 28. Dezember 2016 um 17:53 Uhr

Merry Christmas and a Happy New Year everybody! Thank you for reading this fre:ac status update for December 2016!

It's been a long time since the last update as I've been busy with non-fre:ac related things for the greater part of the year. Thus, progress had been slow until about mid-November, but now we are on track again with two releases in the past four weeks and more to come.

20161129 snapshot release

The 20161129 snapshot mainly fixes an issue on OS X El Capitan that riddled me for almost the whole year.

Soon after the release of El Capitan, users started reporting that fre:ac would hang indefinitely when trying to query the freedb database. Unfortunately, I could not reproduce this on a virtual machine running El Capitan and a friend's MacBook still running Mavericks also showed no issues.

It turned out later that the issue occurred only on multi-core systems (which would probably be almost every system except for my virtual machine). I'm running different versions of OS X in VirtualBox VMs in order to test fre:ac on any OS X from Leopard through Sierra, but unfortunately, VirtualBox does not support more than one CPU core for OS X VMs. That's why that multi-core bug would never happen on my VM...

While trying to reproduce the issue once more in November, I stumbled upon a trick to make multi-core work in VirtualBox. It turned out it's as simple as assigning the VM a pre-Haswell CPUID and it will gladly run with lots of CPU cores. That way I could suddenly reproduce the issue on my own system (and finally run things like parallel conversions on modern OS X too) and fix it after only one or two days.

Besides that fix, the 20161129 snapshot updates codecs to the latest versions and fixes some issues with handling cover art.

fre:ac 1.0.27 and upcoming 1.0.28

After pushing out the 20161129 snapshot, I turned to the 1.0.x branch again and decided to make a new release with some fixes that stacked up in the past months. I originally planned to wait for the release of FLAC 1.3.2 which had been considered on the FLAC developer mailing list in January, but still not been released.

In the meantime (since the fre:ac 1.0.27 release) we finally had some FLAC 1.3.2 preview releases, though, and it looks like it will be ready in a couple of days. As soon as it's finished, I will make a fre:ac 1.0.28 release with the FLAC update and a fix for handling files with broken cover art tags.

Work for the next snapshot

The next snapshot will probably be ready in January and I can already confirm the following changes:

  • Reading cuesheets embedded in tags
    Some programs, like foobar2000, write cuesheets to Vorbis Comment or APEv2 tags when ripping CDs to a single audio file. The next snapshot release will be able to interpret such tags and display the individual tracks when loading such files.
  • More stability fixes
    Another issue affecting mainly the OS X version on multi-core systems has been found and fixed. This made fre:ac prone to crash when interacting with the UI while loading new tracks to the joblist.
  • Haiku port
    fre:ac running on Haiku OSAs a side-project, I ported fre:ac to the Haiku OS, an Open Source reimplementation of the 90ies BeOS operating system.

    Making such ports is fun and often uncovers bugs affecting other operating systems as well. In this case, the aforementioned stability issues on OS X have been discovered mainly because of the Haiku port.

This closes this fre:ac development status update. I will have some additional features and hopefully a new snapshot release to announce in the January issue.

fre:ac development status update 01/2016 Drucken E-Mail
Geschrieben von: Robert   
Sonntag, den 31. Januar 2016 um 22:38 Uhr

Here's the fre:ac devlopment status update for January 2016. Sorry for not giving an update in December! I was so busy with Christmas presents and other stuff that I didn't have much time to work on fre:ac that month.

Updates and fixes

I added a few new features and fixes in the past weeks. Nothing really big, but here's a list of the most important changes:

  • Support for multi-channel Opus
    The 20151122 snapshot supports multi-channel audio with all formats that can handle it, except for Opus. The next shapshot will add that and complete the effort to add basic multi-channel support to fre:ac.
  • Fix for reading floating point samples from .wav and .aiff files
    When reading 32 or 64 bit floating point samples from .wav or .aiff files, the signal is scaled to a range of -1.0 to 1.0 in the current implementation of the SndFile input component. For very quiet recordings, this can lead to a much louder signal after conversion, which was not intended. The next shapshot will fix this.
  • Run fre:ac from a Unicode path on Windows
    Current fre:ac snapshot releases will not start when run from a path that contains Unicode characters on Windows. This is due to a non-Unicode function being used to set the PATH variable in fre:ac's launcher code and will be fixed in the next snapshot.

In addition, I'm currently investigating some issues querying the freedb database on OS X El Capitan. This slowed things down a little in the past weeks, unfortunately, and it's still not fixed. I hope to be able to find a fix in the next one or two weeks.

A DSP engine

The big new feature of the next snapshot release will be the addition of a signal processing engine. This will allow things like sample format conversions, volume normalization and effect filters. The first release with the new engine will have only a handful of DSP components, mainly for sample format conversions, but I'm quite excited about the possibilities.

By the way, the advent of the DSP engine will help close the oldest still open feature request on the SourceForge tracker. In January 2005 a request to add a normalization feature was posted there.

I will probably have more news about this in the next update.

Next snapshot and fre:ac 1.0.27

Integrating the DSP stuff is a lot of work, so I don't think the next snapshot will be out before March. There will be a new release of the stable series in the near future, though. Quite a few fixes have piled up since 1.0.26, so a new release seems justified. I plan to wait for the upcoming FLAC 1.3.2 release, though, in order to have the latest codec versions in fre:ac 1.0.27. The FLAC developers are currently discussing to make a new release on their mailing list.

That's all for this time. Thank you for reading!

fre:ac development status update 11/2015 Drucken E-Mail
Geschrieben von: Robert   
Dienstag, den 01. Dezember 2015 um 18:37 Uhr

Sorry folks, I almost forgot the November status update. Here it is!

November snapshot

I released a new snapshot version a week ago and while most of its new features have already been announced in the October status update, there are a few things that have not been mentioned yet:

  • Setting process priority
    The Windows version of the new snapshot allows you to lower fre:ac's process priority, so it won't slow down other programs too much when running a conversion in the background. This is especially useful on systems with 1-4 cores and no Hyper Threading where fre:ac would usually occupy most of the CPU ressources when converting a large number of files.
  • Recently used output folders
    The configuration dialog now remembers the five most recently used output folders and offers them for selection in a drop down box. If you are often switching between a small number of folders, this should make setting the right output folder much easier.

Stability fixes

Several stability issues have been fixed in November. The following fixes are included in the 20151122 snapshot:

  • There was a chance of the user interface freezing when switching between tabs or dragging the main window during a conversion.
  • A missing lock caused random crashes during conversions in parallel mode.
  • The multiple CDDB match selection dialog could crash due to an issue with loading previews.
  • The freaccmd utility shipped with the October snapshot crashed at startup.

CDDB fixes

In addition to the above, I also fixed a CDDB bug that caused submissions for new CDs to be rejected in many cases. This was caused by a missing break statement in code that checks for disc ID collisions. This bug was introduced in the BonkEnc/fre:ac 1.0 final release almost 9 years ago and was probably one of the oldest bugs still present in fre:ac. I will release fre:ac 1.0.27 with this bug fixed in December; it's already fixed in snapshot 20151122.

This closes this month's update. Thanks for reading!

<< Start < Zurück 1 2 3 4 5 6 7 8 Weiter > Ende >>

Seite 1 von 8

Teilen Add to StumbleUpon Add to diigo