Flac and Audio CD Health Checks
 

In article , Jim Lesurf
wrote:
My next step is to have the program also generate veusz files. That
should make it easier for people to see the results as plots (similar to
what I get with !Tau for RO). Once that's done I'll write a help file
and release the program.

OK, I've now put up a Linux version of Flac_HealthCheck and made it
available from

http://www.audiomisc.co.uk/software/index.html

I've called it version 0.63 to aid 'synching' it with the RO version, but
it's the first Linux version I've produced.

I've included a Help file and the source code. There are also a couple of
webpages

http://www.audiomisc.co.uk/HFN/HealthCheck/CD.html
http://www.audiomisc.co.uk/HFN/HealthCheck/Flac.html

which show examples and explain in more detail how to interpret the
results.

The program is written as a ROX filer application, but can also be used by
execution in the usual ways. It gives output as csv files along with some
'loader' files to quickly plot the results using the Veusz scientific
plotting program. This means if you use ROX and Vesuz then one drag and
drop followed by one click on an icon will show graphs of the results. More
info in the Help file.

Hope people find it useful. Anyone who wishes is welcome ( encouraged! 8-]
) to do versions for other platforms they may use. I know my code is
rubbish, so I'm sure others can do better if they're interested. :-)

I'd say 'enjoy' but the sad fact may be that the results are depressing for
some files.

Jim

--
Please use the address on the audiomisc page if you wish to email me.
Electronics http://www.st-and.ac.uk/~www_pa/Scot...o/electron.htm
Armstrong Audio http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc http://www.audiomisc.co.uk/index.html

Flac and Audio CD Health Checks
 

A question occurs to me prompted by the way the Flack Health Check program
works. (And which I may also use for CD Health Check.)

In operations it relies on writing a tempory file, reading back its
contents, deleting the file, then repeating the process. The 'cycle' takes
somewhat less than a second per tempfile.

This is being written to an ext4 formatted SSD, but could, of course be to
other format/types of storage depending on the user's situation.

Given the bufferring which file systems use, do the files all actually get
written onto the SSD/HD? Or do they go to some kind of ram buffer and are
then deleted again before being actually written to the main part of the
SSD or HD?

Jim

--
Please use the address on the audiomisc page if you wish to email me.
Electronics http://www.st-and.ac.uk/~www_pa/Scot...o/electron.htm
Armstrong Audio http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc http://www.audiomisc.co.uk/index.html

Flac and Audio CD Health Checks
 

Jim Lesurf writes:
A question occurs to me prompted by the way the Flack Health Check program
works. (And which I may also use for CD Health Check.)

In operations it relies on writing a tempory file, reading back its
contents, deleting the file, then repeating the process. The 'cycle' takes
somewhat less than a second per tempfile.

This is being written to an ext4 formatted SSD, but could, of course be to
other format/types of storage depending on the user's situation.

Given the bufferring which file systems use, do the files all actually get
written onto the SSD/HD? Or do they go to some kind of ram buffer and are
then deleted again before being actually written to the main part of the
SSD or HD?

(Depending on mount options) they’ll be buffered at least initially.
Whether the buffer gets flushed while the file is live would be
dependent on timing; I’m not sure if it’ll attempt to flush blocks
corresponding to deleted files if it didn’t flush them when live (you’d
hope not, though).

I’m more confident that there’ll be real writes corresponding to changes
the containing directory.

If your goal is to avoid physical writes if possible, then a tmpfs is a
good bet.

--
http://www.greenend.org.uk/rjk/

Flac and Audio CD Health Checks
 

In article
wwvr3q9q7a0.fsf@l1AntVDjLrnP7Td3DQJ8ynzIq3lJMueXf 87AxnpFoA.invalid,
Richard Kettlewell wrote:

If your goal is to avoid physical writes if possible, then a tmpfs is a
good bet.

Is that something I can expect anyone who used my programs to either
already have, or be happy to setup?

I'd wondered about the above but didn't try it for the following reasons:

RISC OS provides a 'ram' filing system as standard and its reasonably
commonly used. So my RISC OS programs often also use it. Fast and avoids
any 'wear and tear' on hard discs. But there isn't anything obviously
like it on my Mint boxes (or earlier other distros). e.g. no 'icon' on
the backdrop saying 'ram' or 'tempfs' or any hint of a way in. Which
does seem to imply such stuff isn't for ordinary users.

To expand on that: I just checked my (Mint) laptop and it does have a
tempfs set up for /run. And inside that I find, for example, /run/usr/1000
directory that presumably for tempfiles associated with things being run
already for me as user 1000. I can create files, etc, in that when
operating as myself, so don't need to be root. Seems OK for me.

But can I take for granted that the bulk of others who might want to
use my programs also have something like that which I can easily and
reliably use? Ideally for me, a location with the same address for
all users, so I can just use that as the directory to put my tempfiles.

With my RO machines I just use a file with a name like ram::ramdisc0.$.fred
(RO is inherently single user). But what would I do that would 'run
anywhere'? Preferrably without any faffing about probing where a suitable
location might be for a stream of tempfiles.

At present I'm just repeatedly creating a file called (with impressive
originality!) 'tempfile' inside the application's directory. That's
trivially easy for me to do, but risks wear and tear, and may be slower
than a ram based location.

I've done a websearch on this but not found a 'how to' as yet that
makes this simple and clear. So some advice would be welcome as I would
prefer using ram / tempfs if I can understand it, and it's simple
enough for my limited understanding! :-)


Jim

--
Please use the address on the audiomisc page if you wish to email me.
Electronics http://www.st-and.ac.uk/~www_pa/Scot...o/electron.htm
Armstrong Audio http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc http://www.audiomisc.co.uk/index.html

Flac and Audio CD Health Checks
 

Jim Lesurf writes:
Richard Kettlewell wrote:
If your goal is to avoid physical writes if possible, then a tmpfs is a
good bet.

Is that something I can expect anyone who used my programs to either
already have, or be happy to setup?

A common configuration is for /tmp to be a tmpfs. People who don’t make
it a tmpfs presumably don’t care... Put another way there’s a limit to
how much policy your program should be setting; rather, it should be
following the user’s policy.

There are some things to get right if using /tmp.

1. If the environment variable TMPDIR is set, use that instead of /tmp.

2. Don’t use predictable filenames; /tmp is shared between multiple
users. (Get this wrong and you will introduce security problems.)

If you don’t care about the filename, the easiest policy is to use
tmpfile(). That’ll given you an anonymous file somewhere sensible.

If you do care about the filename, use mkstemp().

To expand on that: I just checked my (Mint) laptop and it does have a
tempfs set up for /run. And inside that I find, for example,
/run/usr/1000 directory that presumably for tempfiles associated with
things being run already for me as user 1000. I can create files, etc,
in that when operating as myself, so don't need to be root. Seems OK
for me.

/run isn’t really the right place for application temporary files.

--
http://www.greenend.org.uk/rjk/

Flac and Audio CD Health Checks
 

On Fri, 22 May 2015 13:53:43 +0100, Richard Kettlewell
wrote:

Jim Lesurf writes:
Richard Kettlewell wrote:
If your goal is to avoid physical writes if possible, then a tmpfs is a
good bet.

Is that something I can expect anyone who used my programs to either
already have, or be happy to setup?

A common configuration is for /tmp to be a tmpfs. People who don’t make
it a tmpfs presumably don’t care... Put another way there’s a limit to
how much policy your program should be setting; rather, it should be
following the user’s policy.

There are some things to get right if using /tmp.

1. If the environment variable TMPDIR is set, use that instead of /tmp.

2. Don’t use predictable filenames; /tmp is shared between multiple
users. (Get this wrong and you will introduce security problems.)

If you don’t care about the filename, the easiest policy is to use
tmpfile(). That’ll given you an anonymous file somewhere sensible.

If you do care about the filename, use mkstemp().

To expand on that: I just checked my (Mint) laptop and it does have a
tempfs set up for /run. And inside that I find, for example,
/run/usr/1000 directory that presumably for tempfiles associated with
things being run already for me as user 1000. I can create files, etc,
in that when operating as myself, so don't need to be root. Seems OK
for me.

/run isn’t really the right place for application temporary files.

I don't know if it is the same in all OS, but for temp files I
generally use a word relevant to the programme plus the current
process ID number. This pretty much guarantees a unique name that
won't be used by anyone else.

d

Flac and Audio CD Health Checks
 

(Don Pearce) writes:
I don't know if it is the same in all OS, but for temp files I
generally use a word relevant to the programme plus the current
process ID number. This pretty much guarantees a unique name that
won't be used by anyone else.

Both are highly predictable values. If you’re not using O_EXCL then you
probably have a security vulnerability.

--
http://www.greenend.org.uk/rjk/

Flac and Audio CD Health Checks
 

On Fri, 22 May 2015 14:26:17 +0100, Richard Kettlewell
wrote:

(Don Pearce) writes:
I don't know if it is the same in all OS, but for temp files I
generally use a word relevant to the programme plus the current
process ID number. This pretty much guarantees a unique name that
won't be used by anyone else.

Both are highly predictable values. If you’re not using O_EXCL then you
probably have a security vulnerability.

I wasn't really thinking of security - just avoiding a file name
collision.

d

Flac and Audio CD Health Checks
 

On Fri, 22 May 2015 13:53:43 +0100, Richard Kettlewell
wrote:

There are some things to get right if using /tmp.

1. If the environment variable TMPDIR is set, use that instead of /tmp.

I would add to that, if it isn't set, set it as pointing to /tmp

As a possibly interesting aside which may help some users of embedded
devices, some programs / programming environments have a need or use
of TMPDIR, for example PERL, but this may not always be well
documented. One of the reasons for GetIPlayer or cpan update failing
on embedded devices can be either this variable is not set by the OEM
firmware's default boot up, or else it points to a very limited amount
of on-chip ram. The following sequence of actions in a user post-boot
script can resolve this difficulty, where hard-disk should be
replaced by the normal means of accessing the hard disk of the NAS or
media player in question:
1 if [ ! -d hard-disk/tmp ] ; then mkdir hard-disk/tmp; fi
2 cp -a /tmp/* hard-disk/tmp
3 mount --bind hard-disk/tmp /tmp
4 export TMPDIR=/tmp

In stage 1, you could also have an else statement that cleans out old
files.
--
================================================== ======
Please always reply to ng as the email in this post's
header does not exist. Or use a contact address at:
http://www.macfh.co.uk/JavaJive/JavaJive.html
http://www.macfh.co.uk/Macfarlane/Macfarlane.html

Flac and Audio CD Health Checks
 

In article
wwvlhggrcx4.fsf@l1AntVDjLrnP7Td3DQJ8ynzIq3lJMueXf 87AxnpFoA.invalid,
Richard Kettlewell wrote:

1. If the environment variable TMPDIR is set, use that instead of /tmp.

2. Don't use predictable filenames; /tmp is shared between multiple
users. (Get this wrong and you will introduce security problems.)

If you don't care about the filename, the easiest policy is to use
tmpfile(). That'll given you an anonymous file somewhere sensible.

I'm not clear how that would work for my requirement. I'm using system() to
launch flac and have it put its output into a file which I specify by its
name. Then my main program uses that name to fopen() the file and read from
it.

AIUI tmpfile() will give me a file 'handle' (FILE*) but no name. How would
I use that as part of the command string for flac?

I did find a man page for tmpname() but having given the details it says to
"Never use it!".

Jim

--
Please use the address on the audiomisc page if you wish to email me.
Electronics http://www.st-and.ac.uk/~www_pa/Scot...o/electron.htm
Armstrong Audio http://www.audiomisc.co.uk/Armstrong/armstrong.html
Audio Misc http://www.audiomisc.co.uk/index.html