Location of public APIs

[johnmcfarlane]

I've started writing a Conan recipe in order to cleanly integrate wide-integer into CNL. The plumbing and the conclusions I've come up with are hopefully applicable beyond CNL's concerns so I though I'd share the following observations/suggestions:

• There are two public include directories (the thing that GCC receives as a header search path with options like -I or -isystem):
  • ./math/wide-integer/, and
  • ./util/utility/.
• There are two public global namespaces:
  • util, and
  • math/wide_integer.
• Two of the four headers exposed in public include directories are only necessary for testing, not integration with dependent packages:
  • uintwide_t_examples.h, and
  • uintwide_t_test.h.

In the CMake or Conan scripts, it is possible to shuffle some of the files around in order to omit the test headers from the install destination. However, because of this line

#include <util/utility/util_dynamic_array.h>

two separate install directories are needed and neither -- at their root -- do very much to indicate that they contain headers specific to the wide-integer library. The namespaces suffer from a similar problem: they are one or two levels deep and given very open-ended names at their root: 'util' and 'math'.

Like namespaces, directories (under usr/include at least) are not for creating taxonomies. (That's a great video to watch generally but that nugget of advice translates very well to C++.) By using 'util' and 'math', you're more likely to risk collisions and it's less likely for users to be able to find and remember where you library is on their system. So please consider some minor rearrangement to your source files and their location -- even if you don't go with the following suggestions...

My recommended changes (which are just one possible solution and which I'd be happy to submit in a PR) are to:

• move test-only headers out of the public search path;
• decide on a namespace and a directory for the library (e.g. ::wide_integer and /wide-integer/);
• move public library headers together and away from the rest of the project files, e.g.:
  • ./include/wide-integer/uintwide_t.h, and
  • ./include/wide-integer/dynamic_array.h;
• move the public definitions into this top-level namespace, e.g. ::wide_integer::uintwide_t;
• put non-public (but header-exposed) definitions in a detail sub-namespace, e.g. ::wide_integer::detail;
• move util::dynamic_array and its comparison operators to the detail sub-namespace.

I suggest this with virtually no understanding of:

• the other libraries that might share dynamic_array (there are ways to keep it hidden in detail for wide-integer but inject it into another, public namespace elsewhere), and
• the users of wide-integer who would be disrupted by this change. A less disruptive change is entirely possible but you may find that they appreciate the added clarity -- especially if it is accompanied by build and package management facilities which are simple and idiomatic.

[ckormanyos]

the following observations/suggestions

Thanks for these interesting inputs, John. I think we can find some common compromises, but we need to discuss these just a bit.

Some of these ideas I have also had in similar form(s). These kinds of chasnges could potentially percolate also into my embedded projects extend to some of my other numeric types as well.

I believe the namespace change might be a rather small but breaking change. In the past, I have announced these kinds of breaking changes about a month up front and then implemented them, with no problem really after such things.

[ckormanyos]

Let me wrap my mind around these suggestions and soon get back to you. Then we can decide if the PR is the right place to draft a version adhering to these suggestions, as i agree it probably will be. Then we can see how a working model, let's say, plays out.

[johnmcfarlane]

@ckormanyos no problem. I can work with the multiple directories, headers, namespaces with straight-forward workarounds for now, so no hurry. If CNL comes to depend on wide-integer, they will end up being installed on systems together wherever CNL goes but that's a ways off.

A thought in that direction: while the option of absorbing the algorithms into CNL (as I think you mentioned in another thread) would avoid a lot of complexity, I'm interested in exploring paths which involve maintaining links between the various projects so that fixes and improvements which benefit one project benefit them all.

The application of a build system like CMake and a package system like Conan to a project is one way of making it less painful to plug projects together. Particularly if you're using util_dynamic_array.h in other projects, it's worth knowing that there are ways to reuse this work in a more automated way.

See this PR for latest progress on tieing everything together. This branch is a loooong way from merging!

[ckormanyos]

Well I kind of have to do something. When I first started collecting a few numeric types (independently from my work in Boost), I found myself gathering algorithms and code sequences from as alo as last century.

At first, I put some of them together in rather coherent, yet independent, forms. I now have several nice numeric types (with maybe a few more to come over the years) and several key projects (both published as well as unpublished) that use them.

But it's getting silly with redundancies. I believe my wide-decimal header is used by myself in at least 3 redundant published copies. Also wide-integer is on the same redundant path. Redundancy can be good, it has the advantage that you get only what you need, lightweight, independent and non-dependent on some kind of UNIX philosophy or similar. The disadvantage is redundancy itself, multiple copies, synchronization woes, error propagation and the like.

So I will indeed be working on reducing redundancies and finding more let's say standard storage/distro modes. The tried and true *nix usr/include and urs/lib is a classic --- with local instances of /include and /lib for a given project.

I am a bit stuck on the namespace. I like having the namespace math. But this has come into strong conflict with the popular namespace boost::math so I often end up needing to use my own ::math namespace being resolved with the scope resolution operator. This is kind of inconvenient.

[johnmcfarlane]

Unfortunately, math is not a good choice for the reason you mention and the ones I gave above: just because it's maths-related doesn't lead to wanting it in a "math" namespace. Namespaces are not for taxonomy. It would be better to think up something that's a proper noun.

I started CNL with the name fixed_point, which fails to differentiate it from all the other fixed-point projects. Similarly, I know of a few wide integer projects and I'm sure there are others. By choosing an arbitrary name (like Hana, Clara, Spirit or Boost) or an acronym (like CNL or CTRE) you reduce the likelihood of name collisions (the main reason you'd ever want to use a namespace) and increase usefulness of the name for users.

[ckormanyos]

Hi @johnmcfarlane a very first level simplification could:

• Remove uintwide_t_examples.h and uintwide_t_test.h and place these in the relevant text or examples folder(s).
• Integrate util/utility/util_dynamic_array directly as a file-local detail. This project has no need for a separate utility folder.

I do not think this would influence existing clients of wide-integer. This might not even really help your points in this issue, but does this help?

The potentially ambigous namespace(s) would remain, but single-header only would be the benefit...

[johnmcfarlane]

That sounds like a good plan. TBH I haven't looked at the integration since I started to write my error handling document. I hope to turn back to it soon.

[ckormanyos]

a good plan

Thanks John. OK @johnmcfarlane I took some initial steps, essentially those having no impact on current library use.

• The test/example headers which are, in fact, solely needed for testing have been removed from the main public directory.
• The utility header file <util/utility/util_dynamic_array.h> and its former directory have been removed.
• uintwide_t is now single-header only, dependency-free.

Namespaces and header uintwide_t.h location, however, remain unchanged. The latter would be easier to address in a future change. For the moment, I am a bit reticent to change the existing namespace(s).

[johnmcfarlane]

OK, I'm up to speed with the changes. We can probably resume this conversation. Much has been addressed since the original post. Would you like me to start a new issue with outstanding suggestions or maybe strike through what no longer applies?

Another item I don't think I mentioned is that _t is generally reserved for typedefs. As alluded to in the CompilerExplorer example here, my recommendation would be for two types: wide_integer and integer. These are the latest suggestions from the committee. Certainly uintwide_t now has a misleading u at the start and closely resembles the typedefs from <cstdint> which isn't what that type is about. If they remained a single type, perhaps multiword_integer or something that calls out the multiple limbs.

What are your latest thoughts?

[ckormanyos]

OK, I'm up to speed with the changes. We can probably resume this conversation. Much has been addressed since the original post. Would you like me to start a new issue with outstanding suggestions or maybe strike through what no longer applies?

Yes. I believe we have done well moving forward. Thanks for your help.

Another item I don't think I mentioned is that _t is generally reserved for typedefs. As alluded to in the CompilerExplorer example here, my recommendation would be for two types: wide_integer and integer. These are the latest suggestions from the committee.

Certainly uintwide_t now has a misleading u at the start and closely resembles the typedefs from <cstdint> which isn't what that type is about. If they remained a single type, perhaps multiword_integer or something that calls out the multiple limbs.

What are your latest thoughts?

My unfiltered gut-reaction is to leave the preliminary work of uintwide_t closely as-it-is. A lot of people are using it. Big name changes and redesings of this scale might almost be better done in a new, mostly-independent derivative work that more closely parallels the progress in SG6. I'm just not sure if I would like to metamorphosize uintwide_t --- with all its attributes --- to try to adhere to where we it's going in the committee.

In order to at least add some conformant-look to uintwide_t, we could add some (potentially templated) typedefs that more closely resemble wide_integer and integer.

But if you ask my honest opinion, I feel like the scope of changes being discussed at this point almost need a new derivative work that is more closely resembling a partial reference implementation for wide_integer and integer, which might heavily borrow code or gain inspiration from ckormanyos/wide-integer.

I'm pretty sure the committe needs a reference implementation when dealing with numerics on this complexity level for sure. I would need and expect one. Now just one small part would/could be types like those in my project. I think the drive forward can and should be more closely in line with progress in SG6 --- fully knwoing, ... that is or would be a lot of work.

[johnmcfarlane]

My unfiltered gut-reaction is to leave the preliminary work of uintwide_t closely as-it-is. A lot of people are using it. Big name changes and redesings of this scale might almost be better done in a new, mostly-independent derivative work that more closely parallels the progress in SG6. I'm just not sure if I would like to metamorphosize uintwide_t --- with all its attributes --- to try to adhere to where we it's going in the committee.

If ckormanyos/wide-integer formed the basis for a new type or library, how would support for that work? Would it be a snapshot of wide-integer which was then maintained by me, or by both of us? And would you wish to continue supporting / advancing wide-integer separately?

From CNL's PoV, what happens to ckormanyos/wide-integer is not as important as ensuring we're not expending duplicate effort and that CNL can continue to benefit from the expertise and effort you put into numerics. If I fork wide-integer or snapshot it inside CNL and then wide-integer continues to enjoy bug fixes and improvements from you, then I've created the burden of porting those over. That's really the thing I'm trying to avoid here.

In order to at least add some conformant-look to uintwide_t, we could add some (potentially templated) typedefs that more closely resemble wide_integer and integer.

That would work well except that anyone using those aliases (including CNL users) would also get ::math and ::util added into their global namespace. We could flip your idea on its head and turn uintwide_t into an alias of those new types:

#include <ckormanyos/integer.h>
#include <ckormanyos/wide_integer.h>

namespace math::wide_integer {
  template<const size_t Width2,
           typename LimbType,
           typename AllocatorType,
           const bool IsSigned>
  using uintwide_t = std::conditional_t<
      std::is_same_v<AllocatorType, void>, 
      ::ckormanyos::wide_integer<std::array<LimbType, (Width2 + width_v<LimbType> - 1) / width_v<LimbType>>>, 
      ::ckormanyos::integer<LimbType, std::vector<LimbType, AllocatorType>>>;
}

In this way, wide-integer should remain backward compatible but it would be drawing on the new library. (Note I think I could reduce the number of wide_integer template parameters.)

This is a good choice if you want to keep uintwide_t in active development. But if you're happy to draw a line under it (call it V1) and move on to a new, incompatible interface (call it V2), then it's not necessary because uintwide_t stops being actively worked on.

It's up to you.

[ckormanyos]

If ckormanyos/wide-integer formed the basis for a new type or library, how would support for that work? Would it be a snapshot of wide-integer which was then maintained by me, or by both of us? And would you wish to continue supporting / advancing wide-integer separately?

From CNL's PoV, what happens to ckormanyos/wide-integer is not as important as ensuring we're not expending duplicate effort and that CNL can continue to benefit from the expertise and effort you put into numerics. If I fork wide-integer or snapshot it inside CNL and then wide-integer continues to enjoy bug fixes and improvements from you, then I've created the burden of porting those over. That's really the thing I'm trying to avoid here.

Good points, John. That is, in fact, what I was struggling with in my own thoughts. I thought, ughhh new project, new problems, old project vague support. In these senses it really would make sense to stay consistent with the existing work and evlolve it.

We could flip your idea on its head and turn uintwide_t into an alias of those new types.

Ah yes, of course, that's it. Very stable. Great proposal @johnmcfarlane.

It would avoid breaking change(s), and simultaneously allow for classes of larger/different/more-conformant nature to be built up while fully and actively/dynamically/in-living-project retaining the legacy uintwide_t support if community needs it. I hadn't thought that through yet. Agreed. We should move forward on that line.

We need to take a bit of time, summarize the remaining points of this issue, include the legacy uintwide_t` support and implement the compatibility types --- moving forward.

[johnmcfarlane]

Sounds good. And yes, it won't be trivial, but I think the results will be something many users will be eager to adopt.

For reference, CNL v2 (which is yet to be release) will be a breaking release. That's why I am bumping the big number. I'm sure many users continue to use v1 - if for no other reason then because it supports much older compilers and standards. At such time that they decide to move to C++20, they can then consider moving to v2 but v1 has only fairly minor defects that I know of. This might be an upgrade path to consider offering your users also. (Note that despite the terse C++20 code I'm using to illustrate the proposal, I am not pushing for you to drop C++11 support!)

I've updated the proof of concept here. Note there are fewer template parameters: once you know the container type, then you already know the size/limbs.

[ckormanyos]

Sounds good. And yes, it won't be trivial, but I think the results will be something many users will be eager to adopt.

Yep. Agreed. I've wrapped my mind around it now.

I've updated the proof of concept... Note there are fewer template parameters: once you know the container type, then you already know the size/limbs.

Yes. Great. Let's work together --- in the sense that we have already been doing, or if you like, share the coding even more (all at your option, of course)? There is some significant coding for the first evolutionary step and will take some time.

[johnmcfarlane]

Let's work together --- in the sense that we have already been doing, or if you like, share the coding even more (all at your option, of course)? There is some significant coding for the first evolutionary step and will take some time.

Yes, I recommend small incremental changes to get from where we are to the end state.

First we need to decide what that end-state is. How - if anything - does it differ from the proof of concept? And what are the locations of the files? There is a convention of putting public headers under an /include/ directory (example1, example2). Would the new interface include backward compatibility to uintwide_t? That's a given. Next question, does new API live in ckormanyos/wide-integer repo?

[ckormanyos]

recommend small incremental changes

Yes. For that, I'd like to resolve a question on internal details. If we have a generalized template storage (maybe templated), then do we require internal container storage to have random-access iterators complete with operator[](size_t) on the container members? If so, then the template container choices are limited. If random-access is not required, then I would have to change some internals that use random-access operator[](size_t) to access the values of the member storage variable values.

These would be simple changes, but i'd do these first within the context of uintwide_t before larger desigh refactoring.

Thoughts on container to have random-access or open with only iterator support required?

[johnmcfarlane]

tl;dr probably yes, it requires random access if you're using operator[] but that's no big deal.

It's a good question. You're really the one to answer that because it essentially depends on the algorithms performed over the elements currently.

There's no problem requiring that the container support random access. It is, in fact, tricky to express this in code before C++20 but that's generally not considered too bad a problem: most likely, if the container doesn't have an operator[] then compilation will fail with a somewhat unhelpful error message.

The fact that C++11 is supported doesn't really change this situation. But it does mean that something other than std::array needs to be provided if the user wants constant expressions: std::array does not have good constexpr support before C++20.

Irrespective of this, whenever you can express algorithms in terms of iterators, it's usually a worthwhile abstraction to aim for. It may even help the compiler decide where it can optimise. For example, you could express operator^ in terms of std::transform:

auto operator^(lhs, rhs)
{
  assert(lhs.storage.size() == rhs.storage.size()); // because this is a limited example - not because you cannot do ^ for different width numbers
  Container result(StorageType{lhs.storage.size()});
  transform(begin(lhs.storage), end(lhs.storage), begin(rhs.storage), begin(result), [](auto l, auto r) { return l^r });
  return MyNumberType{result};  // assuming there's a constructor
}

Now, if you're using this approach, that's great... except std::transform isn't constexpr either! IOW, while you're encouraged to avoid [], it's not essential to this refactor and you're up against it anyway because of legacy support.

[ckormanyos]

while you're encouraged to avoid [], it's not essential to this refactor.

I'm giving it a try in a branch --- that means giving it a try to avoid operator[](...) --- and it seems harmless enough with a hundred or two hundred trivial changes.

I'll incubate that and see if it retains speed/passes-tests/embedded-friendly, etc. If that incubates, I'll probably go with iterator-internals, since that's actually the better way to generic programming in the long term.

[johnmcfarlane]

Great! However, this can be pursued in isolation from the kinds of refactors covered by this thread.

At the end of, you may find you can make numbers out of std::list. That would be a gas! 😆

[ckormanyos]

may find you can make numbers out of std::list

That's the plan, maybe not std::list, but I do have some very specialized SRAM-containers and off-chip peripheral containers from the embedded world that would actually greatly benefit from the template container type. Since I want generics here, I decided to push through with the removal of uses of operator[]() random access in the internals. It plays well in the implementation and is in main branch.

updated the proof of concept here

Ah yes. That looks nice. I can see the Boolean parameter. I hope the container of limbs can remain private in the base class.

In the first draft, I might reduce the scope of design to the bounded case and see how that goes. The class design in the proof of concept looks sensible. I will attempt to make a draft and see how it plays. Thanks John!

[johnmcfarlane]

YW. Don't be afraid to add an accessor akin to vector::data():

constexpr Container& data() noexcept;
constexpr Container const& data() const noexcept;

[johnmcfarlane]

Happy New Year! Update: I pretty-much integrated wide-integer into CNL by copy-pasting uintwide_t.h into the CNL repo. Then I've mulled over it. I'm still not comfortable with introducing ::math to my users' programs but I think there's a workaround.

An optional pre-processor macro, WIDE_INTEGER_NAMESPACE, which, if defined wraps all the definitions in

namespace WIDE_INTEGER_NAMESPACE {
...
}

I can then include like so:

#define WIDE_INTEGER_NAMESPACE cnl::_impl
#include "_impl/uintwide_t.h"

It's not a totally trivial change though. There's be a few places in your header where you assume math is global scope. Those'll need fixing.

Thoughts?

[ckormanyos]

I pretty-much integrated wide-integer into CNL

That is great! Thanks for pushing forward on this, John.

It's not a totally trivial change though. There's be a few places in your header where you assume math is global scope. Those'll need fixing.

Thoughts?

I would like to try. I'm wrapping up the clang-tidy checks and I would like to have the bulk of these finished before moving on to the architectural changes.

I like your idea --- in particular the one presented here --- because it allows us to move forward with CNL integration while taking a small-ish step forward in the present architecture.

Let me see how the clang-tidy stuff pans out... Then we can move forward on this idea (or similar forward method as it progresses).

Thank you @johnmcfarlane!

[johnmcfarlane]

I'm happy to continue working on the namespace change. I need to test that it works with CNL anyways.

…

On Wed 5 Jan 2022, 18:38 Christopher Kormanyos, ***@***.***> wrote: It's not a totally trivial change though. There's be a few places in your header where you assume math is global scope. Those'll need fixing. Thoughts? I would like to try. I'm wrapping up the clang-tidy checks and I would like to have the bulk of these finished before moving on to the architectural changes. I like your idea in particular here because it allows us to move forward with CNL integratoin while taking a small-ish step forward in the present architecture. Let me see hpow the clang-tidystuff pans out, then we can move forward on this ides or similar forward methos. Thank you @johnmcfarlane <https://github.com/johnmcfarlane>! — Reply to this email directly, view it on GitHub <#91 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFTGN7CPJV3MICO7OJHR73UUSFZDANCNFSM45MAJ2UA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[ckormanyos]

'm happy to continue working on the namespace change. I need to test that it works with CNL anyways.

Yes, please feel free to forge ahead on that endeavor, John.

Just a head's up... You might see that a rather big refactor is underway for clang-tidy syntax changes. See also #143 and more recently #145.

I don't expect that these cause large merge conflicts, as it's syntax only. But there might be some merging needed depending on when you branch or synchronize things. i will try to wrap up the clang-tidy bulk of the work ASAP.

[johnmcfarlane]

Cool. I've thrown together a PR in #163. LMK what you think.

[ckormanyos]

Hi John (@johnmcfarlane) I am returning to this older issue in order to rekindle our discussion and see what's already done and what next potential steps (if any) could help.

In this thread, we discussed various topics regarding;

1. outer namespace,
2. favoring iterators for internal algorithms,
3. expressing the input template parameter(s) via generic container,
4. anything else I/we missed that could do any possible good for standardization, etc.

Numbers 1 and 2 are finished. I would like to make progress on number 3 and potentially number 4.

Personally, I would like to handle nubmer 3 (potentially in a derivative work that uses uintwide_t. In particular, i would like to use a storage container based on external memory silicon-chips for tiny embedded systems. These kinds of things are in my book on microcontroller programming and having off-chip big integers would be cool for my own future developments in various fields.

Regarding number 4, I do not know if there is anything on your mind that could help, or if synergy is even expected to help. In this matter, I'm happy with wide-integer in its current state. The community seems to be using it and some developers have contributed change requests for new functions that have been done. If, however, there are a few easy adaptions in your mind, we could discuss these in further issues.

Thoughts?

[johnmcfarlane]

Hi Chris, I've been following progress of your library from a distance and occasionally updating my version of the main header. The external memory silicon-chips sounds fascinating but I'm not familiar with that technology. Do you think the iterator approach would help bridge the g with conventional memory representations? If so, I'd recommend exploring that. But regarding CNL integration, I think we're at a good place. As you may have noticed, I've not put much time into the library this year and I have come to the conclusion that standardization of some of the core types would be unlikely, given the current direction the committee is taking. I'm happy to continue discussing design and integration. I'm not sure I'd change my previous suggestions. But I don't want to commit to more effort on CNL right now, so it wouldn't make sense to be deviating wide_int from it's current course. (You're also very welcome to take CNL in new directions of you like.) Re. 3, CNL already wraps wide_int. Are you suggesting a further wrapper, or perhaps an alternative that would alleviate the need for cnl::wide_integer? Re. 4, If you're happy with it, I would proceed as is. I'm pleased to hear it is getting some use it there. And given the way it is integrated into CNL, anyone using the more complex types is already benefiting from it there too. Cheers, John

…

On Sat 1 Oct 2022, 09:39 Christopher Kormanyos, ***@***.***> wrote: Hi John ***@***.*** <https://github.com/johnmcfarlane>) I am returning to this older issue in order to rekindle our discussion and see what's already done and what next potential steps (if any) could help. In this thread, we discussed various topics regarding; 1. outer namespace, 2. favoring iterators for internal algorithms, 3. expressing the input template parameter(s) via generic container, 4. anything else I/we missed that could do any possible good for standardization, etc. Numbers 1 and 2 are finished. I would like to make progress on number 3 and potentially number 4. Personally, I would like to handle nubmer 3 (potentially in a derivative work that *uses* uintwide_t. In particular, i would like to use a storage container based on external memory silicon-chips for tiny embedded systems. These kinds of things are in my book on microcontroller programming and having off-chip big integers would be cool for my own future developments in various fields. Regarding number 4, I do not know if there is anything on your mind that could help, or if synergy is even expected to help. In this matter, I'm happy with wide-integer in its current state. The community seems to be using it and some developers have contributed change requests for new functions that have been done. If, however, there are a few easy adaptions in your mind, we could discuss these in further issues. Thoughts? — Reply to this email directly, view it on GitHub <#91 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFTGNYFRMWZWGMVT2LPIT3WA72DLANCNFSM45MAJ2UA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[ckormanyos]

Thanks for your sage input, as always, John!

external memory silicon-chips sounds fascinating but I'm not familiar with that technology. Do you think the iterator approach would help bridge the g[ap] with conventional memory representations? If so, I'd recommend exploring that

Yes. In fact, I would like to make specialized containers that reside in external memory (off-chip) or also reside within FPGA-memory zones for hardware-acceleration in the future. This is a research topic that interests me. I have previously made memory chunks and std::array-like containers in program-code of certain microcontroller ROM-areas, etc., but not fully made containers that reside in a different random-access physical space (or computational zone) yet. I'll be looking into this, also for faster accelerations, as I feel this topic has not yet been fully handled. This field could work in all kinds of areas, it just happens that wide-integers are very useful for proof-of-concept of such things since an exact, known answer is intuitive and easy-to-get from integral calculations.

regarding CNL integration, I think we're at a good place.

This is great.

so it wouldn't make sense to be deviating wide_int from it's current course

OK. I did actually go entirely in a prototyping run for replacing the internal storage in wide-integer with std::list<limb_type>. This was initially a bit difficult since list does not have random access iteration and my algorithms had, in fact, required index addition/subtraction beyond incrementation. I will probably actually finish a version which is capable of handling internal storage and algorithms with any generic bidirectional iterator (as opposed to requiring random-access-iteration), as long as no performance in the simple random-iteration case is lost. I'll summarize this point more below.

At this stage, I might make one final adaption of the work wich replaces today's AllocatorType parameter with a ContainerType, whereby type definitions (i.e., aliases) maintain the compatibility with the existing code base. I might also elect to not do this. I have not yet decided.

Re. 4, If you're happy with it, I would proceed as is. I'm pleased to hear it is getting some use it there. And given the way it is integrated into CNL, anyone using the more complex types is already benefiting from it there too.

I'm really happy to hear that. So all-in-all, we could actually let this issue settle, possibly close it.

In summary, I will keep today's wide-integer interface as a supported feature. I might adapt to a generic container approach, and I probably will make all internal algorithms bidirectional-iterator-compatible.

The latter point (making all internal algorithms in wide-integer to be bidirectional-iterator-compatible) should be expressed as a separate issue. I'm actually almost done with this in a branch.

[johnmcfarlane]

I'll be sure to keep following with interest. I I presume choice of list was mostly to demonstrate genericity, rather than for efficiency! And I don't see any reason why the APIs you produce couldn't be applied to both fixed-width numbers stored as array, and variable-width numbers across vector. I can even imagine a type backed by deque with an especially efficient operator*=()! All of this should be possible so long as the underlying algorithms have the right interface. Naturally, if I were going to design the API for the new type, it would closely resemble the wrapper exposed in CNL. Regardless, I'm sure it will find wide applicability. (Excuse the pun.)

…

On Mon 3 Oct 2022, 08:33 Christopher Kormanyos, ***@***.***> wrote: Thanks for your sage input, as always, John! external memory silicon-chips sounds fascinating but I'm not familiar with that technology. Do you think the iterator approach would help bridge the g[ap] with conventional memory representations? If so, I'd recommend exploring that Yes. In fact, I would like to make specialized *containers* that reside in external memory (off-chip) or also reside within FPGA-memory zones for hardware-acceleration in the future. This is a research topic that interests me. I have previously made memory chunks and std::array-like containers in program-code, etc., but not fully made containers that *reside* in a different physical space (or computational zone) yet. I'll be looking into this, also for faster accelerations, as I feel this topic has not yet been fully handled. This field could work in all kinds of areas, it just happens that wide-integers are very useful for proof-of-concept of such things since an exact, known answer is intuitive and easy-to-get from integral calculations. regarding CNL integration, I think we're at a good place. This is great. so it wouldn't make sense to be deviating wide_int from it's current course OK. I did actually go entirely in a prototyping run for replacing the internal storage in wide-integer with std::list<limb_type>. This was initially a bit difficult since list does not have random access iteration and my algorithms had, in fact, required index addition/subtraction beyond incrementation. I will probably actually finish a version which is capable of handling internal storage with any generic bidirectional iterator (as opposed to requiring random-access-iteration), as long as no performance in the simple random-iteration case is lost. At this point, I might make one final adaption of the work wich replaces today's AllocatorType parameter with a ContainerType, whereby type definitions (i.e., aliases) maintain the compatibility with the existing code base. I might also elect to not do this. I have not yet decided. Re. 4, If you're happy with it, I would proceed as is. I'm pleased to hear it is getting some use it there. And given the way it is integrated into CNL, anyone using the more complex types is already benefiting from it there too. I'm really happy to hear that. So all-in-all, we could actually let this issue settle, possibly close it. In summary, I will keep today's wide-integer interface as a supported feature. I *might* adapt to a generic container approach, and I probably will make all internal algorithms bidirectional-iterator-compatible. The latter point (making all internal algorithms in wide-integer to be bidirectional-iterator-compatible) should be expressed as a separate issue. I'm actually almost done with this in a branch. — Reply to this email directly, view it on GitHub <#91 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFTGNYCZGUJHXLEFP4LXX3WBKD3PANCNFSM45MAJ2UA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[ckormanyos]

choice of list was mostly to demonstrate genericity, rather than for efficiency!

Oh yes, absolutely. The experience using std::list revealed that I had used random-access iteration in nearly 200 (more than I had thought!) places within the thousands of lines in wide-integer. I thus did a rather massive refactor of these to handle bidirectional iteration.

At the end of refactoring I had not lost any performance for random-access iterators. So that was cool. This work does, however, need to settle for a while and I still need to optimize some more areas for the generic case.