Is version 5.0 in development? #109
-
|
I heard that version 5.0 of the software is in development to be released soon. I don't want to upgrade my version from 4.2 to 4.24 if 5.0 is on the horizon. Is there an estimated release date the developers can let us know about? Thank you! |
Beta Was this translation helpful? Give feedback.
Replies: 6 comments 7 replies
-
|
Zach, It's Mark Taylor at UDOT. This is a good question so let me be as specific as I can with version 5.0:
There are probably a few other things that will get done in version 5.0 (if not 5.0 probably 5.1 shortly thereafter):
If you have a functioning ATSPM system, I would suggest to wait for 5.0. It is actively being worked on by two developers. I promise that in the future, we will do much smaller releases so sorry about the long wait. -- Mark Taylor |
Beta Was this translation helpful? Give feedback.
-
|
Hi @taylormd3 , No pressure, but just wanted to check-in to see if you had any idea on a release date for version 5.0. We are debating internally on rebuilding our entire instance of ATSPM to clear up some issues and wanted to hold off on the latest version to start from scratch. Thanks, |
Beta Was this translation helpful? Give feedback.
-
|
Zack, The most recent release of ATSPM is version 4.2.5. We're currently wrapping up version 4.3 that we have been working on for at least a year now - if not longer. We're beta testing the last little bit of it. I anticipate us completing the beta test probably by the end of next week (we added some scope creep to the development that slowed things down). We've been beta testing version 4.3 for the last month. Once we complete the beta testing on V 4.3, we will release it on GitHub. I estimate 4.3 will be on Github sometime this month. Version 4.3 has lots of improvements, including:
VERSION 5.0 |
Beta Was this translation helpful? Give feedback.
-
|
@taylormd3 , thanks for the update. Will version 4.3 contain fixes for communicating with and correctly downloading/deleting the .DATZ files from Econolite controllers? That's one of the major issues I'm facing right now and I have to run a separate script to handle Econolite controllers that run a newer firmware. Or will I need to wait until version 5.0 for that to be corrected? Either way, it sounds like we'll need to plan on getting to version 4.3 when it's released. |
Beta Was this translation helpful? Give feedback.
-
|
I don't know about the problem you're talking about. We just run one program for Econoite... But in v5.0 there is a completely new signal controller logger with a different structure that is more flexible. When we get closer you can try it out and see if it works for you. |
Beta Was this translation helpful? Give feedback.
-
|
@udotdevelopment , there is definitely a problem with my version of ATSPM not being able to delete the .DAT/DATZ files off the econolite controllers that are running newer firmware versions. So it keeps all the files on the controller and downloads them all again every 15 minutes, causing our database to grow insanely fast. I had to script a solution to delete the files off the controller because the ATSPM executables fail to. This is what I get logged for the errors in the database
|
Beta Was this translation helpful? Give feedback.
-
|
With the newer Econolite firmware that generates .datZ files, we shouldn’t be doing the old hack of turning off and on the data logger. The controller keeps the active file as .dat and then compresses to .datZ at the end of the interval when a new file is generated. Therefore, files don’t need to be deleted. We need logic to ignore files already processed so data doesn’t get duplicates.
Sent from Yahoo Mail for iPhone
On Thursday, June 9, 2022, 12:29 PM, ZachParkerCR ***@***.***> wrote:
@udotdevelopment , there is definitely a problem with my version of ATSPM not being able to delete the .DAT/DATZ files off the econolite controllers that are running newer firmware versions. So it keeps all the files on the controller and downloads them all again every 15 minutes, causing our database to grow insanely fast. I had to script a solution to delete the files off the controller because the ATSPM executables fail to. This is what I get logged for the errors in the database
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
I disagree, for a couple of reasons.
The hack of turning off and on the data logger was to work around the logger stopping if the program deleted the current file. There are better solutions to that problem now, like using smaller log files and we can distinguish the active log from the completed ones more easily. But, that didn't have much to do with the need to delete the existing logs from the controller. (And considering the years it takes for some agencies to update controllers, we still need the toggle hack)
The internal log management of the ASC/3 and Cobalt controller has been, historically, unreliable. It is not uncommon for the controller to fill up the available storage space and then cease to log anymore, no matter what the retention settings were on the controller. Deleting the collected logs avoids that problem.
Additionally, it is a really easy way to avoid collecting duplicate logs. Yes, we can keep track of the files we have collected, and not collect them again, but that is more meta-data management that I don't think is really worth the pay-off. We can query the events table for the most recent record and only collect logs later than that, or keep a record of the last collection time from each device and use that timestamp as a filter. That isn't a bad solution, but the time differences between the controllers and the ATSPM server are going to leave gaps or duplicate rows. Every step we add to the process, even if it is a small one, will add to the time it takes to complete the task. Parallel processing can only get us so far, and the system has to collect and store logs from every device connected to it faster than the logs are written.
ZachParkerCR, you should investigate why the delete task is failing. There might be errors in the database or written in the console of the FTP program that would give us a clue.
From: Steve Gault ***@***.***>
Sent: Thursday, June 9, 2022 4:04 PM
To: udotdevelopment/ATSPM ***@***.***>
Cc: Subscribed ***@***.***>
Subject: Re: [udotdevelopment/ATSPM] Is version 5.0 in development? (Discussion #109)
With the newer Econolite firmware that generates .datZ files, we shouldn't be doing the old hack of turning off and on the data logger. The controller keeps the active file as .dat and then compresses to .datZ at the end of the interval when a new file is generated. Therefore, files don't need to be deleted. We need logic to ignore files already processed so data doesn't get duplicates.
Sent from Yahoo Mail for iPhone
On Thursday, June 9, 2022, 12:29 PM, ZachParkerCR ***@***.***<mailto:***@***.***>> wrote:
@udotdevelopment , there is definitely a problem with my version of ATSPM not being able to delete the .DAT/DATZ files off the econolite controllers that are running newer firmware versions. So it keeps all the files on the controller and downloads them all again every 15 minutes, causing our database to grow insanely fast. I had to script a solution to delete the files off the controller because the ATSPM executables fail to. This is what I get logged for the errors in the database
-
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.Message ID: ***@***.***<mailto:***@***.***>>
-
Reply to this email directly, view it on GitHub<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fudotdevelopment%2FATSPM%2Fdiscussions%2F109%23discussioncomment-2917604&data=05%7C01%7CShane.Johnson%40q-free.com%7C831d6b90d2704e63b54508da4a640800%7C39ef44caadfa4a248858ef68dbe16345%7C1%7C0%7C637904090747182294%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=6ZqQZEJtA87w3InVnJNnVZW7YkICQOlTc%2BRzLV8B3IE%3D&reserved=0>, or unsubscribe<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAHPHYUTDGY5QIDSCNLMKQMDVOJS6XANCNFSM5YD3ZCKA&data=05%7C01%7CShane.Johnson%40q-free.com%7C831d6b90d2704e63b54508da4a640800%7C39ef44caadfa4a248858ef68dbe16345%7C1%7C0%7C637904090747182294%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=urkxBaz45n0d7RZD%2F0YRCYFAXOFPlxFr9GTzVv3ipRg%3D&reserved=0>.
You are receiving this because you are subscribed to this thread.Message ID: ***@***.******@***.***>>
|
Beta Was this translation helpful? Give feedback.
-
@shanejohnson-atspm We just decided internally to turn off the scheduled tasks so my database guy can clean up the billions of records of duplicates and then I'm having our traffic staff revert our Econolite controllers back to the previous firmware that didn't have issues deleting the files from the controllers. Once I get the environment stable, I'll be turning the application tasks back on to see how things are functioning. |
Beta Was this translation helpful? Give feedback.
-
|
The two firmware versions that pose a problem with allowing the files to be deleted using the ATSPM processes are 32.67.20 and 32.67.30.
Zach Parker
Systems Analyst – Information Technology
City of Cedar Rapids
500 15th Ave SW, Cedar Rapids, IA 52404
Office: (319) 286-5642
Email: ***@***.******@***.***>
From: UDOT ***@***.***>
Sent: Monday, June 13, 2022 10:06 AM
To: udotdevelopment/ATSPM ***@***.***>
Cc: Parker, Zach M. ***@***.***>; Author ***@***.***>
Subject: Re: [udotdevelopment/ATSPM] Is version 5.0 in development? (Discussion #109)
CAUTION:This email originated from outside the organization. Do not click links or open attachments unless you know the content is safe. Never provide your user ID and password!
What is the firmware version of your controllers?
—
Reply to this email directly, view it on GitHub<#109 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AMQRI53HALOAWZRBQP375LLVO5E4RANCNFSM5YD3ZCKA>.
You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>>
[http://cedar-rapids.org/Email-Graphic-Customer-Service-3.png] <https://survey123.arcgis.com/share/709743d7252949e68ae788238416d91d>
|
Beta Was this translation helpful? Give feedback.
-
|
We at UDOT are currently using the following firmware versions (due to limitations mostly with our traffic signal central system (of what is supported with that), as well as what works with ATSPM logging: Cobalt: 32.67.20 Q-Free (Intelight): 2.5.3 ASC3: Boot - 1.14.03; App - 2.62.10 |
Beta Was this translation helpful? Give feedback.
-
|
@taylormd3 , if you're on Cobalt 32.67.20, did you have to do any customizations to ATSPM to get those controllers to correctly process the FTPFromAllControllers application without error? At first, I couldn't get files downloaded from the controllers on that firmware version because apparently, the path changed on the controllers where the log files exist. So I had to create a new controller type in the database and adjust the path commands so that when the FTP process connects, it starts in the correct folder. Once I had that working, the FTP process was still not deleting the files from the controller, so it would continuously collect a ton of log files and pull them all down every 15 minutes, creating a lot of duplicates. I had to script a separate process that ran WinSCP commands to properly connect to the controllers, download the files, and then send a delete command to make sure they didn't stay on the controller. But not all of our controllers are on the 32.67.20 firmware - most are on 32.65.00, which doesn't have the problem with file deleting. So I have to have the ATSPM FTP process running alongside my custom script, which also ends up causing duplicates because two FTP processes are running on the controllers with newer firmware. So if you did something custom in your environment, I'd love to hear what you suggest for mine. Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
@ZachParkerCR - I'm not sure what we did to make it work and believe we have kept those versions as they are fully compatible with our version of MaxView. A lot will be changing in the next few months - 1) We will be migrating over to Kinetic signals from MaxView and 2) Our developers are working on ATSPM version 5 that will most likely change how we pull the files back from the controllers. At that time, we will most likely upgrade our controller firmware (or consider it - assuming there are benefits we get from the controller by upgrading). |
Beta Was this translation helpful? Give feedback.
Zach, It's Mark Taylor at UDOT. This is a good question so let me be as specific as I can with version 5.0:
5.0 is a big release (bigger then we generally like to do with releases). We don't have a release date yet but we're getting closer. I know it's a bit vague but we're getting very close to being done with two very large projects within version 5.0. a) Left Turn Gap Analysis Report and b) Data Compression. A third very large project (15-minute data aggregation files) has already been completed.
If I were to guess on an anticipated date (and it is a guess), I would think that 5.0 will be released probably around early spring. If you're already at version 4.2, I would just wait for…