Implement IRUS-US statistics #765
Comments
Now the repository is viewable again in the browser
|
Merged
|
Partially resolved with #766 The maven build seems ok: The jar seems to be there: |
|
In the DSpace log, I see The problem, then, is related to “Usage Statistics are not logged if usage-statistics.dbfile is not configured, or if IP is not mappable to a location #2965” - DSpace/DSpace#2965 IP address 172.20.7.12 is a private address and therefore not mappable to a location. Is it the address of a proxy? If so, it needs to be configured to forward the actual client IP (assuming that is locatable). - Paul |
|
Turning on Debugging (optional) If you'd like to try and do some debugging yourself, you can change the DSpace logger settings to DEBUG which will sometimes provide you with more information about the error. To turn on debugging, visit the [dspace]/config/log4j.properties file and do the following: To enable DEBUG logging in the dspace.log file, change the log4j.rootCategory and log4j.logger.org.dspace settings to DEBUG rather than INFO. (XMLUI Only) To enable DEBUG logging in the cocoon.log file, change the log4j.logger.org.apache.cocoon setting to DEBUG rather than INFO. NOTE: You'll need to restart Tomcat after enabling DEBUG mode in the log4j.properties file. Note: changing log4j.logger.org.apache.cocoon to debug causes something similar to DS-239: java.net.MalformedURLException: unknown protocol: resource, so I only changed log4j.rootCategory and log4j.logger.org.dspace settings to DEBUG. |
|
The VT Libraries uses a private IP network for staff including 172.20.7.12. @soumikgh tested it with another IP address in the VT IP range but not in the private network. When he used that the do not appear. |
|
To get around the private IP issue in my LDE, I used curl with X-Forwarded-For to request the item and a bitstream, log: and log: Attached In this log, this gets rid of the IP location error. I see the LoggerUsageEventListener, GoogleRecorderEventListener, and the SolrLoggerUsageEventListener but the atmire debug messages do not appear. |
|
I have reported this issue in DSpace 6.3 IRUS patch troubleshooting #7 in the Atmire DSpace IRUS-UK patch repository. |
|
@keithgee built an LDE at his remote location and the tracker is working with expected DEBUG messages. He temporarily opened his IP and JISC reports seeing activity from his IP in the logs for their test server. He tested retry failed trackings by disconnecting his wireless. This created the error in the log, This produced the entry in the database, Next, he reconnected wireless and ran It didn’t empty the table though, which it should have. He even tried changing the table from owner postgresql to dspace. |
|
Since @keithgee 's LDE works, I suspect that it is due to his having an IP outside of the VT Libraries subnet. At VT Libraries, I did try both the VPN and Eduroam wireless with my iMac, but still no tracker DEBUG messages. At home with my home ISP, I created a new Vagrant box and ran tail. There were no debug messages from atmire when I selected items and files. We have compared installation method, Vagrantfile, site_secrets, and database backup. The mystery continues. |
|
Let me give you a summary of what I did: (1) Installed the patch for dspace 6.3 from https://atmire.github.io/IRUS/#/prerequisites (2) I had to make an additional change to the patch because the IPs IRUS was getting were not of the users, but of our server: In this file: dspace/dspace/modules/atmire-statistics-exporter/atmire-statistics-exporter-api/src/main/java/com/atmire/statistics/expor/ExportUsageEventListener.java In this method: Make this change: (3) After building, in config/modules/stats.cfg
I was not sure what your present issue is from reading the link you sent, but could what I did in (2) help? Let me know if this helps! -Jose, U. Michigan |
|
I created 3 LDEs here and got the atmire DEBUG messages without doing anything special. However, I was testing on the third LDE this morning and it worked for a few minutes, but then stopped working. This was just a few minutes ago and I haven't been able to make it work again. |
|
We temporarily disabled the VT restriction on dev and the DEBUG atmire messages did not appear. Further testing ideas:
|
My third LDE started working again after I halted it to make sure that the second LDE was still working, and then restarted it. |
|
@keithgee, would you upload a DEBUG level log for a bistream download to match the one I uploaded for I'd like to see where the two logs diverge. Thanks. |
|
@alawvt I can do this in a bit, but I don't have this bitstream in any of my LDEs. So I expect the IRUS stats code will not run because of an error before that time. I'll give it a shot and see what happens. |
|
Oh, you can load any test file. I don't think that will matter. But it's here, https://vtechworks-dev.cloud.lib.vt.edu/handle/10919/103119. |
|
@alawvt I've emailed a log file to you. I discovered something. Remember when I said that uploading a bitstream broke the IRUS stats on my LDE earlier? I thought it happened again just now, but not quite, in both cases. When I'm logged in, the IRUS stats are not reported. When I'm not logged in, they are reported. Do you know if this is by design? I'm examining the code for the supplied patch now, so that may shed some light on it, but I'm wondering if this is a bug or a feature. Edit: Logged in = logged in on the VTechWorks web interface. |
|
Thanks for the log file. That's really interesting. Are you saying that the log event is recorded after you log out of your computer or out of VTechWorks on your LDE or something else? |
|
Here are the steps.
Also, I will send you a new log file in a bit. The one I just sent was me clicking. I think what you actually wanted was for me to run that curl command. |
|
in case this is still useful : |
|
@keithgee, @davidatmire left a very useful comment in atmire/IRUS#7 (comment), that traffic from logged in admins is not tracked. |
|
@keithgee, thank you very much for the logs. Comparing the logs for downloading a bitstream on your LDE to my laptop using eduroam, I see: Yours: Mine: Observations:
Do you have the GeoLite database installed? Searching through these logs, I see that yours has 36 occurrences of while yours has none. |
|
On dev, dspace.log.2021-09-29 contains: which matches the log from my LDE. |
|
No, Geolite database is not installed on any of my LDEs |
|
@keithgee and I built new LDEs on our laptops with |
|
in you LDE, in dspace/config/modules/usage-statistics.cfg, what is there for usage-statistics.dbfile? Is it commented out? It’s worked for me with it commented out, with an entry there but the file missing, and with the entry there and the file in place. The latest LDE has |
|
I created another LDE on my MacBook and deleted the database in it with Then I packaged the vagrant box with Note: The IP for this is 192.168.60.5 Following the instructions in https://scotch.io/tutorials/how-to-create-a-vagrant-base-box-from-an-existing-one, I downloaded the package to my iMac in new folder, alaw_no_IRUS_20211002/. and ran no site on http://192.168.60.5/ still no site on http://192.168.60.5/ The Vagrantfile is very bare bones and doesn't include the install directory, IP address. I will have research how to export that. |
|
With Vagrantfile, Vagrant up builds and I loaded the db. But it gives a 404. |
|
@keithgee built the machine directly, without using Vagrant and it, too, wored for the atmire stats. I decided to try this myself. I updated the instructions in vtlibans readme - Option 2: Redeploy dspace changes, building with maven on local computer and rsync to vm. I ran to the existing LDE on my iMac. Alas the stats debugging messages do not appear. |
|
I built dspace manually on my iMac and the atmire debug messages appear. Now I can compare it to the one in the LDE. The log shows the two Atmire debug messages immediately preceeding the other messages that appear in my LDE. |
|
For debugger Intellij IDEA, The DSpace server built ok with the the IDEA Community Edition. To add the Tomcat server we'll need the IDEA Ultimate edition (30 day trial). |
|
We won't continue this project. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
From: Hannah Rosen
Date: Mon, Aug 23, 2021 at 10:31 AM
Subject: IRUS implementation instructions
To: Philip Young
Hi Philip,
I hope you are well. We are working on the invoice for your IRUS service. In the meantime, you can start implementing the tracker on your repository.
I think that is it. Let me know if you have any questions!
Best,
Hannah
Implement - Software Platforms - DSpace
We have Tracker patches, developed by Atmire, for the three most recent versions of DSpace. To be able to install a patch there are the following prerequisites:
The patches are available on Github (atmire/IRUS)
The text was updated successfully, but these errors were encountered: