constellation overlay adjustment using a good reference image

[morrowwm]

I've been thinking of a new way to align the constellations. Could you give some advice on whether this will work or not?

The attached zip file contains a primitive html file which loads the included image and overlays it with a virtualsky starmap for the same time and my location. The two files should be written to your allsky/html/allsky directory, so they pickup the virtualsky.js library.

The image is from my camera, and shows a real nice view of the Big Dipper, Polaris and some other bright stars.

My thought is to add some GUI controls to adjust the overlay (rotation, panning, scaling) until is matches the underlying image. virtualsky supports rotation using the mouse or keyboard shortcuts, but not scaling without callback to the server.

The resulting adjustments after the overlay is aligned should be convertible to allsky's setting to align the constellation overlay, and should be faster than the current trial and error approach.

My javascript skills are pretty limited, so feel free to hijack this idea. :)
constellation_experiment.zip

[EricClaeys]

@morrowwm, not ignoring you, just have been busy.

[morrowwm]

Perfectly understandable, @EricClaeys. I’m working on the idea. I’m impressed with your diligence (and others) in supporting the community over the last few days.

I'm confused by one concept in the constellation overlay documentation:

Most allsky cameras produce rectangular pictures, but the portion of the picture with your allsky image is often round, surrounded by black. Further, the round portion may not be centered exactly in the picture, although it's usually close.
Later, the "round portion" is called the lens image.
...
Determine the center of the lens image (not the center of the whole image).

Is this center the pixel which the sky is rotating about? I.e. Polaris for the northern hemisphere? I don't think so. To me it seems, it's always the exact middle of the image. How could it be different? It the lens is misaligned on the camera sensor?

And one other question: does the constellation overlay assume the camera is perfectly flat, i.e. looking straight out from its latitude and longitude?

[troublestarter]

Haven't succeed to configure mine.
i'm waiting for best day without rain to do it.

But sure, it's a good idea...

[aaronwmorris]

I believe the "center of the lens image" is the center of the image circle. It is extremely common for the sensor to not be perfectly aligned with the lens.

An offset of 0.1mm will shift the circle about 25 pixels on a camera with 3.75um pixels. 65 pixels on a camera with 1.55um pixels (HQ camera).

[Alex-developer]

^^ This :-)

I guess you could work out the center of the image circle, either with some funky coding or get the user to do it, then adjust the origin of drawing the overlay.

Or allow the user to drag/rotate the overlay around the canvas and resize it until it fits, this would be pretty easy to do

[EricClaeys]

@morrowwm, If you can think of better wording for the first part, please do so.

here's a brain dump of what it's supposed to say is (I'm in a hurry so don't have time to think much):

• Almost all allsky camera sensor are rectangular.
• The image produced by a lens is always round (should be consistent with what it's called - maybe "round lens image?)
• The round image may not be exactly centered in the rectangular image, but will usually be close. (one of my cameras has a lens that's slightly off center).
• a. The round lens image may totally fit on the rectangular sensor, which results in black borders.
• b. The round lens image may instead be larger than the rectangular sensor which results in what looks like a "regular" image.
• c. The round lens image may instead partially fit the rectangular sensor which results in black borders on the sides but not top and bottom.

Probably should have a picture of the three cases above.
The center of the round lens image is the physical center, regardless of what's in the image. The sky rotates around Polaris (more or less) which will never be the center of the round lens image unless the lens is pointing directly at Polaris which should only happen if you live at the north pole.
Most cameras point straight up so the center of rotation is much lower than the center.

I'll have to think about your final question. My gut says the software doesn't care where the cameras is pointing, although not pointing straight up will probably introduce funky distortions which will make it harder to align things.

[morrowwm]

OK, thanks for all that. My lens seems to be completely covering the sensor, i.e. case "b" above. Here's an over-exposed full resolution image of another cloudy, rainy night.

I've been hurting my brain reading this: https://math.stackexchange.com/questions/4780406/finding-the-transformation-matrix-that-maps-one-2d-plane-in-3d-space-to-another

I think it might be the seed for what I'm contemplating.

[morrowwm]

As an experiment, I hard coded a transformation to align constellations to my image right in the polar projection function of virtualsky, replacing:

    this.projections = {
        'polar': {
            title: 'Polar projection',
            azel2xy: function(az,el,w,h){        
                var radius = h/2;
                var r = radius*((Math.PI/2)-el)/(Math.PI/2);
                return {x:(w/2-r*Math.sin(az)),y:(radius-r*Math.cos(az)),el:el};
            },

with

    this.projections = {
        'polar': {
            title: 'Polar projection',
            azel2xy: function(az,el,w,h){
                // hardcode transformation to camera coordinates
                scaling = 3.1;
                rotation = 1.95;
                xoffset = w/2 - 95;
                yoffset = h/2 - 265;
                    
                var radius = yoffset;
                var r = scaling*radius*((Math.PI/2)-el)/(Math.PI/2);
                x=r*Math.sin(az);
                y=r*Math.cos(az);
                xprime = xoffset + x*Math.cos(rotation) + y*Math.sin(rotation);
                yprime = yoffset - x*Math.sin(rotation) + y*Math.cos(rotation);

                return {x:xprime,y:yprime,el:el};
            },

It works well. Below is a highlight of the transformed overlay over my original image. I've inverted the overlay (press "i") to avoid drowning out the image.

If I can learn javascript better, I could add controls to vary the scaling, rotation, xoffset, yoffset parameters on the web page. Once they are satisfactory, put them in allsky configuration.

I am not a fan of forking this code, though. Is it actively worked on, or just maintenance? I gathered the latter from comments in its repository.

[EricClaeys]

@morrowwm, I like what you are doing. In the prior Allsky version I was very hesitant to make any changes to virtualsky, but I noticed it didn't appear to be supported so in the current release I made more changes, although I added comments with ALLSKY next to the changes.
If your test works I would be happy to include it in Allsky as long as you can add an on/off statement in the code that can be toggled via a new field in configuration.json. Unless of course your code can replace the existing code in all cases.

[morrowwm]

@EricClaeys or someone, could you point me at some tutorial material for the flavor of javascript we use? It's angular? I'm relatively clueless at javascript, so it's slow going learning it by just reading the allsky code.

This? https://www.w3schools.com/angular/angular_intro.asp

[Alex-developer]

Its all vanilla JS using jquery, no angular

Alex

[spikked27]

Any progress on this? I just spent 2 hours trying to align the overlay and it's still not great, a tool for alignment would be an absolute dream

[morrowwm]

I've made some progress.

Struggling with how to pass changed virtualsky configuration so that it redraws properly.

[EricClaeys]

I've attempted to get resizes of the constellation overlay to work; it's better, but still not perfect. I've probably made it way more complicated that it needs to be.
If you find a better have feel free to fix.

[morrowwm]

@EricClaeys, you mean in allsky, correct? I've seen some modifications tagged with ALLSKY in virtualsky.js.

I'm going to spend some time trying to better understand how virtualsky works.

[EricClaeys]

@morrowwm, yes, in allsky. The ALLSKY tags in the virtualsky/* code are what I added.

[morrowwm]

Gotcha. I have made decent progress. Try it at https://eastmorrow.ddns.net/allsky/aligner.php

In my scratch repository https://github.com/morrowwm/allsky-aligner aligner.php, I call into the virtualsky object that is global in the script which does the alignment controls. So I think it's the same object, i.e. does not get destroyed/constructed constantly?

For example:

	else if(navElement === "togglestars") {
		planetarium.toggleStars();
	}	

is what is done if you click on the star button. I'm using some fonts that might not be on every platform

virtualsky.js is a bit ... javascript-y. I have not made it any better. It's possible the way I am calling it would simplify things little. I need to finish the alignment functionality, then figure out how to get the results

Would we ever use any projection other than polar or fisheye?

[EricClaeys]

@morrowwm,
Glad to hear you are making progress. Javascript isn't my strength, especially with constructors/destructor and anything object oriented.

Would we ever use any projection other than polar or fisheye?

I can't think of any for a typical allsky lens with a 120 degree or wider FOV.

I'll look at you site in the next day or two.
Thanks for doing this.

[morrowwm]

LOL, should I use this? https://www.npmjs.com/package/nudged

1. present the real sky and mis-aligned sky map
2. user clicks "References" button, then clicks on a few reference points. In my test image, Polaris, Mars, Jupiter, Aldeberan, say.
3. user clicks on "Sources" button, then clicks the same points as they are positioned on the sky map
4. The page then uses this nudged javascript library to estimate the optimal transformation to align the skymap using rotation, translation and scaling

const transformationMatrix = nudged.estimate({
  estimator: 'TSR',
  domain: sourcesky,
  range: referencesInImage
})

5. the transformation can be used everywhere before an (x,y) pair "point" is drawn on the canvas:

pprime = nudged.point.transform( point, transformationMatrix)

I think I'll get the manual method more polished first.

The way this is headed, we may want to modify the way allsky draws the overlay. I'll wait until I get there.

[morrowwm]

I would appreciate it if a couple of people could test out https://github.com/morrowwm/allsky-aligner with their setup.

It picks up your latitude, longitude from allsky's configuration.json plus some other constellation overlay settings. So there is minimal configuration required.

The README.md of the above repository intends to explain how to install it. Please save a copy of your baseline virtualsky.js!

My explanation on how to get the test image loaded are not lucid.

This only handles "projection": "fisheye" so far. If you want to test and use "projection": "polar", let me know and I will implement that.

[Alex-developer]

I would appreciate it if a couple of people could test out https://github.com/morrowwm/allsky-aligner with their setup.

It picks up your latitude, longitude from allsky's configuration.json plus some other constellation overlay settings. So there is minimal configuration required.

The README.md of the above repository intends to explain how to install it. Please save a copy of your baseline virtualsky.js!

My explanation on how to get the test image loaded are not lucid.

This only handles "projection": "fisheye" so far. If you want to test and use "projection": "polar", let me know and I will implement that.

This has been on my todo list for ages so glad someone has got something working, it looks great.

I will give it a try and let you know how I get on

Alex