Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make it easier to reprocess images on other machines #478

Merged
merged 6 commits into from
Sep 16, 2021
Merged

Make it easier to reprocess images on other machines #478

merged 6 commits into from
Sep 16, 2021

Conversation

ckuethe
Copy link
Collaborator

@ckuethe ckuethe commented Sep 14, 2021

rm -rf /tmp/nonexistent doesn't fail on nonexistent files, so unconditionally nuke the sequence directory

$ rm -rf /tmp/nonexistent && echo that worked
that worked

provide some default values for the framerate and encoder, just in case you don't have them in your configs.

`rm -rf /tmp/nonexistent` doesn't fail on nonexistent files, so unconditionally
nuke the sequence directory

```
$ rm -rf /tmp/nonexistent && echo that worked
that worked
```

provide some default values for the framerate and encoder, just in case you
don't have them in your configs.
@EricClaeys
Copy link
Collaborator

Since $ENCODER isn't used in timelapse.sh, we should probably change this line:
-vcodec libx264 \
to:
-vcodec ${ENCODER} \

Also, to be consistent, we should add
TIMELAPSE_BITRATE=${TIMELAPSE_BITRATE:-2000k}
below the new ENCODER=${ENCODER:-libx264} line
and MOVE
TIMELAPSEWIDTH=${TIMELAPSEWIDTH:-0}
below the new ENCODER=${ENCODER:-libx264} line

Further, change:
-b:v ${TIMELAPSE_BITRATE:-2000k} \
to
-b:v ${TIMELAPSE_BITRATE} \

This will keep all the default settings that may not be in config.sh at the top of the file together.

@ckuethe
Copy link
Collaborator Author

ckuethe commented Sep 14, 2021

Whoops. forgot to push that commit. Anyway, you see where I'm going with this ... Ideally I want to be able to run timelapse.sh on my workstation and offload the movie making to CUDA.

@EricClaeys
Copy link
Collaborator

Understand. By putting all the settings that would normally be in config.sh at the top, it makes it clearer what you'd need on your workstation.

Now this sort of thing works:
```
~$ env FFLOG=info VCODEC=h264_nvenc ~/allsky/scripts/timelapse.sh 20210913 /nas/home/allsky/20210913/
timelapse.sh: Creating symlinks to generate timelapse
```
Makes tuning encoder settings a little easier by keeping the sequence.
Not meant for general consumption which is why it's only documented in
a comment.
```
~/allsky$ env VCODEC=h264_nvenc ~/allsky/scripts/timelapse.sh 20210913 /nas/homes/allsky/20210913/
/home/ckuethe/allsky/scripts/timelapse.sh: line 7: /home/ckuethe/allsky/config.sh: No such file or directory
jq: error: Could not open file : No such file or directory
timelapse.sh: file EXTENSION not found in configuration, guessing jpg
timelapse.sh: Creating symlinks to generate timelapse
timelapse.sh: Created 6340 links total
...
```
@EricClaeys
Copy link
Collaborator

Chris, in timelapse.sh, should the line -vcodec ${VCODEC:libx264} \ be -vcodec ${VCODEC:-libx264} \ ?

@EricClaeys
Copy link
Collaborator

Chris, it would be nice if timelapse.sh had some comments at the top saying there is code added to allow it to run on a different computer from the Pi, and then describe what the environment needs to include (variables, etc.). Things like guessing the EXTENSION aren't needed on the Pi since it's in filename.sh.
I was a little confused looking over the code until I realized what some of it was for.

Also, in timelapse.sh the statement NSEQ=$(ls "${DIR}/sequence" | wc -l) should probably be NSEQ=$(ls "${DIR}/sequence" 2>/dev/null | wc -l ) to avoid an error message when the "sequence" folder doesn't exist (it usually won't).

@ckuethe
Copy link
Collaborator Author

ckuethe commented Sep 15, 2021

As far as I can tell, I'm the only person ever who wanted to reprocess frames into new videos on a much more powerful machine. The default behavior of timelapse.sh is fine for the common use case - running it on the allsky host from endOfNight.sh - but I have a couple years worth of captures and it's nice to be able to offload those to CUDA, potentially with some extra filters.

Remember, what I'm aiming to simplify here is the workflow of checking out allsky from git, then being able to run something like:

cd /nas/home/allsky/
for d in * ; do
    env VCODEC=h264_nvenc ~/allsky/scripts/timelapse.sh $d /nas/home/allsky/$d/
done

Note that I'm not specifying extension or frame rate, nor have I edited config.sh... That's why I'm guessing the extension if unspecified.

Really, I'd like to rewrite most of the support scripts in python. It takes 25-30 to start allsky on my pi due to the loop that computes capture arguments with jq. Python can do that computation in 1.5ms on my pi, and even the whole python start, module import, json parse, commandline generation, and output takes less than 320ms. Large speedups are also possible when creating the frame symlinks. My pi can do over 1325 symlinks/sec on a usb stick.

Copy link
Collaborator

@EricClaeys EricClaeys left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thomas,
These changes look good to me and are working on my Pi.

@EricClaeys
Copy link
Collaborator

Chris,
FYI, few other people have expressed interest in running timelapse generation on a more powerful machine with more memory. Hence the reason for my last message.
We can always add instructions for doing so to the Wiki if needed.

@thomasjacquin thomasjacquin merged commit ed4f615 into AllskyTeam:master Sep 16, 2021
@ckuethe ckuethe deleted the tweak_timelapse branch September 16, 2021 18:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants