Skip to content
This repository has been archived by the owner on May 21, 2024. It is now read-only.

OTA-2536: Preconfigure IpUptaneSecondary with IP address #1183

Merged
merged 1 commit into from
Apr 18, 2019

Conversation

mike-sul
Copy link
Collaborator

  1. Tiny IpUptaneSecondary refactoring
  • to make it working with a fake package manager;
  • to get serial & hw-id from Secondary during initialization;
  1. Pre-configure Secondary with IP addresses by using a json configuration file;
  2. Initialize Aktualizr app with Secondaries.

Signed-off-by: Mike Sul [email protected]

@OYTIS
Copy link
Contributor

OYTIS commented Apr 17, 2019

This pull request introduces 1 alert when merging f5b19d3 into 99bfcf9 - view on LGTM.com

new alerts:

  • 1 for FIXME comment

Comment posted by LGTM.com

@mike-sul mike-sul force-pushed the feat/OTA-2536/preconfigure-ip-secondaries branch from f5b19d3 to 87325ea Compare April 17, 2019 15:25
@codecov-io
Copy link

codecov-io commented Apr 17, 2019

Codecov Report

Merging #1183 into master will decrease coverage by 0.79%.
The diff coverage is 6.52%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master    #1183     +/-   ##
=========================================
- Coverage   77.91%   77.12%   -0.8%     
=========================================
  Files         170      173      +3     
  Lines       10003    10114    +111     
=========================================
+ Hits         7794     7800      +6     
- Misses       2209     2314    +105
Impacted Files Coverage Δ
src/libaktualizr-posix/ipsecondarydiscovery.cc 78.57% <ø> (-0.74%) ⬇️
src/libaktualizr/uptane/secondaryconfig.h 100% <ø> (ø) ⬆️
src/libaktualizr/config/config.h 100% <ø> (ø) ⬆️
.../aktualizr_secondary/aktualizr_secondary_common.cc 88.23% <ø> (ø) ⬆️
src/libaktualizr-posix/asn1/asn1_message.cc 20.4% <0%> (ø) ⬆️
src/libaktualizr/uptane/secondaryinterface.h 83.33% <0%> (-16.67%) ⬇️
src/aktualizr_secondary/aktualizr_secondary.cc 21.87% <0%> (-0.47%) ⬇️
src/libaktualizr-posix/asn1/asn1_message.h 74.07% <0%> (-2.85%) ⬇️
src/libaktualizr-posix/ipuptanesecondary.cc 1.21% <0%> (+1.21%) ⬆️
src/libaktualizr-posix/ipuptanesecondary.h 0% <0%> (ø) ⬆️
... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e3f0242...e37947f. Read the comment docs.

@OYTIS
Copy link
Contributor

OYTIS commented Apr 17, 2019

This pull request introduces 1 alert when merging 87325ea into 99bfcf9 - view on LGTM.com

new alerts:

  • 1 for FIXME comment

Comment posted by LGTM.com

Copy link
Collaborator

@pattivacek pattivacek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good so far. I haven't reviewed everything carefully yet, and the ASN1 stuff I'm too unfamiliar with to offer any substantial advice on.

using SecondaryFactoryRegistry =
std::unordered_map<std::string, std::function<std::shared_ptr<Uptane::SecondaryInterface>(const SecondaryConfig&)>>;

static SecondaryFactoryRegistry sec_factory_regisrty = {
Copy link
Collaborator

@pattivacek pattivacek Apr 17, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: sec_factory_registry.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

AKDiscoveryReqMes ::= SEQUENCE {
port INTEGER(0..65535),
...
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not critical at all, but the indenting here is inconsistent.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if I see it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you look at the surrounding lines on github, it is more apparent. I think it's a tab vs. spaces thing.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, a tab is used for indentation here while my editor inserts spaces on tap pressing.

explicit IpUptaneSecondary(const sockaddr_in& sock_addr, EcuSerial serial, HardwareIdentifier hw_id,
PublicKey pub_key);

// what this method for ? Looks like should be removed out of SecondaryInterface
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's used by ManagedSecondary. It's possible that it could be merged into the constructor, but I think there used to be a good reason it was necessary before. It may not be necessary now. Not priority, though. We can look into that further once more of the refactoring is done.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok.

void Initialize() override{};

// It looks more natural to return const EcuSerial& and const Uptane::HardwareIdentifier&
// and they should be 'const' methods
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might be right. In the past we didn't do that in case system calls or other non-const-friendly operations were required to implement these. We can re-evaluate that again soon.

@@ -37,16 +37,18 @@ class SecondaryConfig {
std::string ecu_public_key;
KeyType key_type{KeyType::kRSA2048};

// FIXME: secondary config contains secondary specific params
// so introduction of any new type of secondary will require changes here/libaktualizr
// what we would like to avoid
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, this is not ideal. It might be a necessary evil for now, but we should try to remove them if possible. Not the top priority, though.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK

@@ -20,21 +20,32 @@ namespace Uptane {

class SecondaryInterface {
public:
// This ctor should be removed as the secondary configuration SecondaryConfig
// is the secondaries's specific, see SecondaryConfig declaration
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are probably right. I think a lot of this was done to make SecondaryFactory work smoothly, and since we are removing that, we can get rid of some of the associated cruft along with it.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. I can try it in the next PRs since it leads to to many changes across the source code.

virtual void Initialize(){}; // optional step, called after device registration
// should be pure virtual, since the current implementation reads from the secondaries specific config
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that makes sense.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll try in the next PR.

virtual Uptane::HardwareIdentifier getHwId() { return Uptane::HardwareIdentifier(sconfig.ecu_hardware_id); }
virtual PublicKey getPublicKey() = 0;

// getSerial(), getHwId() and getPublicKey() can be moved to seperate interface
// since their usage pattern differ from the following methods' one
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Separate interface? What do you mean?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, serial, hw_id and pub_key are constant properties of a secondary and can be determined/obtained before or during a secondary object instantiation, while all other methods represent RPC calls that actually transfer messages between Primary and Secondary. I don't think, that there is any practical need/benefit in breaking the current interface into two parts because of the aforementioned, but it might appear in the future.

@mike-sul mike-sul force-pushed the feat/OTA-2536/preconfigure-ip-secondaries branch from 87325ea to 6aa63eb Compare April 17, 2019 18:18
Copy link
Collaborator

@pattivacek pattivacek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few more thoughts after I had more time to review. Still looking pretty good on the whole, I think.

@@ -42,6 +43,7 @@ bpo::variables_map parse_options(int argc, char *argv[]) {
("primary-ecu-serial", bpo::value<std::string>(), "serial number of primary ecu")
("primary-ecu-hardware-id", bpo::value<std::string>(), "hardware ID of primary ecu")
("secondary-configs-dir", bpo::value<boost::filesystem::path>(), "directory containing secondary ECU configuration files")
("secondary-config-file", bpo::value<boost::filesystem::path>(), "secondary ECUs configuration file")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just noting that we should get rid of secondary-configs-dir once we are able to. I think we have to leave both for now, but we should eventually convert virtual secondaries (and anything else still depending on the old factory mechanism) to use the new secondary-config-file.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, fully agree.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be helpful to have more different naming and avoid confusion in the meantime (haven't had a great idea though). Would it also make sense to disallow using both options at once?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll remove secondary-configs-dir soon (in one of the following PRs)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. I haven't had time to look the details in depth but is the new format really providing the old functionality?
In particular, can virtual secondaries be still used? Some docs refer to them and they are used by qa and more for testing.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The plan is to adjust virtual secondaries implementation to the new configuration approach.


if (sec_cfg_factory_registry_.find(secondary_type) == sec_cfg_factory_registry_.end()) {
LOG_ERROR << "Unsupported type of sescondary config was found: `" << secondary_type
<< "`. Ingoring it and continue with parsing of other secondary configs";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: "Ignoring". And actually "continuing" would be better than "continue".

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.

if (socket_fd < 0) {
throw std::system_error(errno, std::system_category(), "socket");
}
// what's the point in usage of SocketHandle
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no idea.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll get rid of it in the next PR.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it automatically closes the socket, once it's out of scope.

Copy link
Collaborator Author

@mike-sul mike-sul Apr 18, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, your're right. Normally, such RAII objects imply object initialization/creation and deinitialization/deletion at ctor and dtor correspondingly. Also, the dtor's part should include shutdown and preferably verification if a socket was actually opened.

Asn1Message::Ptr Asn1Rpc(const Asn1Message::Ptr& tx, const struct sockaddr_storage& client) {
int socket_fd = socket(AF_INET6, SOCK_STREAM, 0);
Asn1Message::Ptr Asn1Rpc(const Asn1Message::Ptr& tx, const struct sockaddr_in& server_sock_addr) {
int socket_fd = socket(AF_INET, SOCK_STREAM, 0);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AF_INET6 is no good?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Benefit of ipv6 usage is questionable in this case (local network within a car).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are the drawbacks?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well,

  1. people get used to IPv4 addresses, so easier to edit a configuration file (::1 instead of 127.0.0.1 and IPv6 slightly longer
  2. At the kernel level (IP stack implementation) slightly more work for packets processing

We consider IPUptaneSecondary as a reference implementation, so I tend to make it simple, unless there is a requirement to support IPv6

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually have no strong feelings here. I'm fine with keeping it simple for now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really buy the performance considerations if not tied to a specific platform/system architecture. Disabling v6 support also seems easier than adding it after the fact to a v4 only code base.

That said, I would have hoped that we wouldn't have to worry about either v4 or v6 by using dual stack sockets. If that's not the case and makes development harder, then ok for dropping it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's worth keeping in mind this whole functionality is just meant to be a reference and demonstration; so it is perfectly acceptable to make a semi-arbitrary choice and just roll with it.

Copy link
Contributor

@eu-siemann eu-siemann Apr 18, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair point, I forgot that it will actually support both AF_INET and AF_INET6. Then I'm for leaving it as it was before :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@patrickvacek sure, sounds reasonable

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous version supported only AF_INET6 and it's used the 'discovery' mechanism to get IPv6 addresses. Since, we are going to remove the 'discovery' mechanism and input IP addresses in the config file then there is no need to stay with IPv6.
I prefer to choose either IPv4 or IPv6, adding support of both will require additional effort, but I am fine with either choices.


req->present(AKIpUptaneMes_PR_publicKeyReq);
EcuSerial serial{"serial"};
HardwareIdentifier hw_id{"hw_id"};
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd suggest initializing these to empty strings. Otherwise it might be confusing in error conditions. Especially for testing, we often use dummy values that might be easily confused with these defaults.

Copy link
Collaborator Author

@mike-sul mike-sul Apr 18, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried, but there is an input string length verification in EcuSerial ctor what doesn't allow me to specify an empty string.
Can I remove it as someone removed it for HardwareIdentifier ?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I remember running into that before, and somehow I was able to work around it. That seems less possible here. We'd need to do a quick check that removing that restriction won't open up any holes, but it's probably fine.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like remove it is fine, at least tests passes on my local env.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was fairly sure the tests would pass, but I mean we should look at where EcuSerials are instantiated and check if we need to add any new checks for non-emptiness in those places.


for (auto& config : secondary_configs) {
try {
LOG_INFO << "Creating Secondary of type: " << config->type();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"\n"?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually LOG_XXX adds "\n" to the end of a message, so I am removing "\n" I added in other places.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

huh, good to know

@@ -61,6 +61,15 @@ void SocketServer::HandleOneConnection(int socket) {
// Figure out what to do with the message
Asn1Message::Ptr resp = Asn1Message::Empty();
switch (msg->present()) {
case AKIpUptaneMes_PR_getInfoReq: {
std::cout << ">>>>>>>>>>>> Got Info request";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Debug leftover?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes.

@mike-sul mike-sul force-pushed the feat/OTA-2536/preconfigure-ip-secondaries branch from 6aa63eb to e37947f Compare April 18, 2019 11:15
Copy link
Collaborator

@pattivacek pattivacek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this looks fine unless there's still something you wanted to clean up in this PR.

@mike-sul
Copy link
Collaborator Author

I think this looks fine unless there's still something you wanted to clean up in this PR.

No, I am fine with it.

@pattivacek pattivacek merged commit f5bb1cc into master Apr 18, 2019
@pattivacek pattivacek deleted the feat/OTA-2536/preconfigure-ip-secondaries branch April 18, 2019 14:21
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants