proposal: Go 2: support read-only and immutable values in Go

[zigo101]

The full proposal is here: https://github.com/go101/immutable-value-proposal/blob/master/README-v9.1.md

Basically, this proposal can be viewed as a combination
of issue#6386
and issue#22876.

This proposal also has some similar ideas with
evaluation of read-only slices written by Russ.

[beoran]

Reading the proposal, I kind of like the idea of having final values, which are immutable var values. But I don't like the idea of having to annotate my function parameters with .reader and .writer everywhere. The compiler should be smart enough to the right thing without annotations.

[zigo101]

The compiler should be smart enough to the right thing without annotations.

How to do it without any hints for compilers? I think there must be some hints for compilers to prevent some behaviors.

BTW, the syntax is :reader, not .reader, and there is not :writer.

[zigo101]

I do think the :reader part is less necessary than the final part in Go.
Not supporting read-only package-level values in Go is a more acute problem.
Maybe this proposal can be simplified as final values are also reader values,
by discarding the :reader syntax part.

[beoran]

I think so, after all a final value is read only by definition. Please consider updating your proposal with this in mind.

[zigo101]

It is not that simple. Final values are also reader values has its only drawbacks and problems. The complexity in the current one has its rationals. I will try to make a new one for final values are also reader values when I think it clearly.

[romshark]

Hi, author of proposal: Go 2: immutable type qualifier here!

First, I'm glad to hear people talk about this topic more and more! Go lacks important compiler-enforced safety features and while I can live without compile-time memory safety (O&B) - living without read-only types is dangerous. My proposal is currently on hold since I don't have much time investigating the const-poisoning problem which is a blocker and I hope you heard of it.

1. a final is immutable, except that it's not?!

Please note that, although a final itself can't be modified, the values referenced by the final might be modifiable. (Much like JavaScript const values and Java final values.)

JavaScript's const qualifier is the worst safety feature I've ever seen! I'd even go as far as calling it an "anti-safety feature" because it's very misleading and error prone! It makes const x look like a constant, but when it's a reference or includes references - it's totally mutable. An object of type T is what it's made of, including member references, so if you're saying that x in final x T is immutable while its members like x.y are not - it's misleading and dangerous IMO.

I've gone the other way in my proposal: an immutable type makes all its members immutable, which make all their members immutable, recursively. If you see var x immut *T you'll know for sure that x won't be able to mutate any referenced memory, ever.

2. The more exceptions there are to a rule - the worse the rule is.

:reader is not allowed to appear in type declarations, except it shows up as function parameter and result roles.

IMO, in an imperative (non-functional) language mutability is a property of types, not a property of values. A type may reference a value but it can't protect it from other mutable types referencing it. Go is not a functional language, it can't guarantee value immutability (FP-style immutability can have significant performance impacts and would be unacceptable in case of Go), it can only protect memory from being mutated through certain references ultimatelly making the code easier to read and debug. The only immutable values in Go are constants, but those are primitive-types only for good reasons.

If immutability is a property of types, then why can't we make this rule universally applicable to all types without exceptions?

3. final and :reader imply transitive immutability, which is what I was initially trying to avoid for a reason.
The mechanism I proposed is very flexible, you can define immutable slices of immutable pointers to mutable values: var immut []* mut T, you can define mutable pointers to immutable values: var * immut T and so on. I don't see this possibility with final and :reader because they're applied to variables rather than types.

What happens, if the developers needs this kind of fine-grained control over what's mutable and what's not in a type-chain? Exactly, he/she will throw immutability out the window if it's too limiting and this is obviously not what we'd want to happen I suppose.

A safety-feature that covers only simple cases but fails to cover rather complex cases isn't a safety feature.

4. The :reader syntax doesn't look promising
Even if you make immutability a property of types it still doesn't particularly look very sexy IMHO:

// proposal 31464:
var x []:reader *:reader T:reader
var y []:reader * T:reader

// proposal 27975:
var x immut []*T
var y immut [] mut * immut T

// proposal 31464 (with MQP):
var x []*T:reader
var y []:reader *:writer T:reader

BTW, I called this feature Mutability Qualification Propagation, it reduces verbosity quite substantially.

P.S.
I apologize if I happened to misinterpret your proposal because I only had so much time to glance through it quickly.

[zigo101]

immutable and read-only are different. An immutable can't be modified in any (safe) routines., but read-only values can. Final values themselves are immutables, but the values referenced by them might not. In fact, the values referenced by a final value might be a read-only value, an immutable value, a writable value, depend on how this final value is declared. An example:

var s = []int{1, 2, 3}
final x = []int{1, ,2 ,3}:reader // the elements referenced by x are immutable
final y = s:reader               // the elements referenced by y are read-only (through y).
final z = s                      // the elements referenced by z are writable (through z)

Please note the y example is like the one var x immut *T your mentioned in your proposal.

I agree JavaScript's const qualifier is bad, which is why I think final values need :reader values to cooperate to satisfy all kinds of needs.

IMO, in an imperative (non-functional) language mutability is a property of types, not a property of values.

I agree mutability can be viewed as a property of types, but I think read-only is mainly a property of values. As I have said in my proposal, Although roles are properties of values, to ease the syntax designs, we can think they are also properties of types.

If immutability is a property of types, then why can't we make this rule universally applicable to all types without exceptions?

The reason is the following notations are all invalid:

[]([]int:reader):reader
*(T:reader):reader
struct {
    x T:reader
}

// including the ones you made:
[]:reader *:reader T:reader
[]:reader * T:reader
[]:reader *:writer T:reader

A reader value represents a read-only value chain, so only one :reader is needed.

... which is what I was initially trying to avoid for a reason. The mechanism I proposed is very flexible, you can define immutable slices of immutable pointers to mutable values: var immut []* mut T, you can define mutable pointers to immutable values: var * immut T and so on

This is what my proposal tries to avoid, for this brings much complexity and such needs are rare in practice.

And I'm some confused by your description. In your first example, var x immut *T, you say x won't be able to mutate any referenced memory, ever. So you mean var x immut *T is equivalent to var x immut * immut T, instead var x immut * mut T? If true, then no confusion for this now. In my opinion, the syntax is too verbose.

The :reader syntax doesn't look promising

Sorry, the following notations are invalid by my proposal.

// proposal 31464:
var x []:reader *:reader T:reader
var y []:reader * T:reader

they should be:

// proposal 31464:
var x []*T:reader
var y []*T:reader

And there is not a :writer notation, the following one is impossible by my proposal.

var y []:reader *:writer T:reader

[romshark]

There won't be any "real" immutability in Go because it's not a functional language, actual immutability can only be achieved in languages that don't support the assignment operator. Introducing immutability would imply introducing PDS (Persistent Data Structures) to improve performance, otherwise we might end up having to copy the hell out of every final.

Final values themselves are immutables, but the values referenced by them might not.

I'm strictly against such kind of magic, because it's misleading and error prone! It makes a variable look immutable, but it doesn't make it immutable because a variable value isn't just the address of the referenced data, it's the data itself (recursively).

var s = []int{1, 2, 3}
final z = s // the elements referenced by z are writable (through z)

Actual immutability is a nice concept (especially for application programming), it makes code readable and safe, but since Go is a general purpose non-FP language and does support assignments, hence it's better to leave real immutability out of Go to keep it simple and consistent and stick to read-only types IMHO. This way you could, for example, write your own PDS libraries using read-only types (in combination with generics this could be great) but the language itself doesn't force you to follow any particular concept.

I refer to "immutable" when a value is referenced by read-only references exclusively and is thus virtually immutable (you can only modify it through unsafe pointers, but this is okay, we can't forbid people to shoot themselves in the foot, it's called "unsafe" for a reason). If you want to make a variable read-only then make it have a read-only type and use mut -> immut casting:

var s = []int{1, 2, 3}
var x = immut []int {1, 2 ,3} // the elements referenced by x are immutable
y := immut []int(s)           // the elements referenced by y are read-only (through y).
h := [] immut int(s)          // the elements are writable, but the slice is not.

IMO this is better, because it's much more readable, you can see what's mutable and what's not, you don't have to assume:

w := &T{}             // mutable
r := immut *T(w)      // write-protected (recursively)
v := immut * mut T(w) // address is read-only, but the underlying data is writable

I agree mutability is a property of types, but I think read-only is not. Instead, read-only is a mainly a property of values. As I have said in my proposal, Although roles are properties of values, to ease the syntax designs, we can think they are also properties of types.

Again, Go is a non-FP language, there won't be immutable values, because it's fundamentally imperative and supports assignment. You could mutate a seemingly immutable value through unsafe pointer and all hell will break loose. We can't make Go an FP language and we shouldn't try to make it pseudo-FP. There won't be immutable values in Go.

This is what my proposal tries to avoid, for this brings much complexity and such needs are rare in practice.

That's where our opinions differ. I'd prefer a flexible tool, that could safe me in complex situations (but I'd still try to avoid complex situations as much as possible of course), rather than a toy, that's okay for 90% of cases but practically useless in the really difficult 10%, this ain't a safety feature then IMO.

And I'm some confused by your description. In your first example, var x immut *T, you say x won't be able to mutate any referenced memory, ever.

x is a read-only pointer to a read-only instance of T, that's correct because...

So you mean var x immut *T is equivalent to var x immut * immut T, instead var x immut * mut T? If true, then no confusion for this now. In my opinion, the syntax is too verbose.

This is called MQP (Mutability Qualification Propagation). MQP reduces verbosity, but gets rid of transitive qualifiers. It may not be obvious at the first glance but it's very simple: a mutability qualifier propagates to the right in a type chain, until it's canceled out by another qualifier:

// with MQP
var p immut *T
var m immut map[string][]string
var x immut [][] mut T

// without MQP
var p immut * immut T
var m immut map[immut string] immut [] immut string
var x immut [] immut [] T

Sorry, the following notations are invalid by my proposal.

I got that, that's why I explicitly stated: "Even if you make immutability a property of types it still doesn't particularly look very sexy IMHO:". But I should probably have written: "Even if you remove transitive qualification"

And there is not a :writer notation, the following one is impossible by my proposal.

I know, it was just a hypothetical example.

[zigo101]

Again, Go is a non-FP language, there won't be immutable values, because it's fundamentally imperative and supports assignment. You could mutate a seemingly immutable value through unsafe pointer and all hell will break loose.

Yes, we can use unsafe do many things to break the type system. The fact has already existed, so it is not a drawback of this proposal.

Imperative immutability is a real need of many Go programmers. It is very useful.

I think you have got it clearly what are the differences between the two proposals. The main intention of my one is to keep both the syntax and concepts simple, which sticks to Go style.

Let's agree on what we agree and disagree on what we disagree. :)

[romshark]

Yes, we can use unsafe do many things to break the type system. The fact has already existed, so it is not a drawback of this proposal.

Sure, I just wanted to emphasize that in an imperative language where assignment is allowed - immutable values don't exist by nature. There may be read-only references and virtually immutable objects (referenced by read-only references exclusively), but no actually immutable values/memory.

Imperative immutability is a real need of many Go programmers. It is very useful.

I do absolutely agree. Makes code safer and APIs clearer. It does have some drawbacks like const-poisoning which I couldn't yet investigate but it's probably solvable.

I think you have got it clearly what are the differences between the two proposals.

feature	#31464	#27975
concept	final and :reader	immutable types
qualification	transitive qualification applied on variables	mixable qualification applied on types with MQP
immutable struct fields	no?	yes (immutable after initialization)
read-only methods	yes	yes

Those aren't all differences I suppose, but I didn't have the time to investigate it in full detail yet. Please correct me if there's something wrong/missing.

The main intention of my one is to keep both the syntax and concepts simple, which sticks to Go style.

Those are good intentions, but I disagree on the way you're trying to achieve them. IMO immutable/read-only types are easier and way more consistent than having two concepts final and :reader.

There's also some inconsistencies like:

• We can't send values to final channels.
• We can't receive values from final channels.

Why? we already have read-only channels in Go. Why do we need yet another variant?

[zigo101]

Something to note:

• MQP also applies to reader values.
• fields of final structs are immutable struct fields.

Why? we already have read-only channels in Go. Why do we need yet another variant?

It is not a special rule. It is just a consistent general rule for all kinds of types in this proposal, it is not channel specified. It is just that final values are not modifiable, whatever their types are. There are not no-directions channels, right? Final is a stronger property than single-direction.

[romshark]

MQP also applies to reader values.

What do you mean by that? MQP is necessary for mixed mutability type chains to reduce verbosity. If I understood correctly there's no mixed qualification with :reader but transitive qualification instead.

fields of final structs are immutable struct fields.

What I'm talking about is this:

type User struct {
  Name      immut string
  BirthDate immut *time.Time
  Devices   immut [] mut Device
}

u := U{
  Name:     "initial name",
  BirthDate: nil,
  Devices:   []Device{Device{Name: "initial name"}}
}

u.Name = "new name"                       // compile-time error!
u.Devices[0] = Device{Name: "new device"} // compile-time error!
u.Devices[0].Name = "new device name"     // OK

Final values themselves are immutables, but the values referenced by them might not.

If final only applies to the variable but not necessarily its referenced contents then why can't we read/write a final channel? A channel is just a reference to an internally shared thread-safe queue, it's a reference type. I'd rather assume that c in final c chan int is just not reassignable, but still writable/readable.

[zigo101]

What do you mean by that? MQP is necessary for mixed mutability type chains to reduce verbosity.

OK, I misused it. It should be IMQP, immutability Qualification Propagation. :)

u.Devices[0].Name = "new device name"

I deliberately discarded the partial read-only/immutability feature. Please read the end of my proposal for details. This feature brings many confusions and complexities.

A channel is just a reference to an internally shared thread-safe queue. A channel is just a reference to an internally shared thread-safe queue, it's a reference type.

I don't like the reference type terminology. It brings many confusions. But here I use it for your convenience. It is true that a channel is just a reference to an internally shared thread-safe queue. Our dispute is that whether or not the internal queue should belong to the channel itself or to the value the channel references. I choose the former interpretation, you choose the latter. I think my choice is better. Not only do I think the internal queue should belong to the channel itself, the elements stored in the queue (or received from and sent to the channel) should also belong to the channel itself. That is it.

[zigo101]

We should inspect channels in logic, not in the internal implementation. Channels are different to slices and maps.

For example, a chan int value doens't reference any value, but a []int value references some int values.

[ianlancetaylor]

Thanks for the detailed proposal.

The :reader and :writer syntax just isn't Go like. There are no similar constructs in the language.

:reader and :writer seems to be used with both values and types, but it's not clear what that means. I'm also unclear on whether a :reader type can only be assigned to another :reader type; that is, are these type qualifiers that must appear on all uses of the type?

It's not obvious to me how final works in a case like final sub = s[:5] where the backing array of the slice is shared with a mutable value.

The number of extra rules necessary to make this work, even assuming that it is complete, seems very large compared to the benefit to the rest of the language.

The problem area remains interesting to investigate, but we aren't going to adopt this proposal.

[zigo101]

@ianlancetaylor
I think it is not a good idea to close this issue when there are so many questions you still have.

There is not the :writer notation in this proposal. It is welcome to suggest an alternative for the :reader syntax. The syntax is not the core of this proposal.

I'm also unclear on whether a :reader type can only be assigned to another :reader type; that is, are these type qualifiers that must appear on all uses of the type?

I don't very understand this question. Could you show an example to help me understand it?

The number of extra rules necessary to make this work, ...

I don't think the number of extra rules in this proposal is large. The rule set is quite small IMHO.

It's not obvious to me how final works in a case like final sub = s[:5] where the backing array of the slice is shared with a mutable value.

In fact, I have an alternative proposal which makes a little modification on this one.
If you re-open this issue, I will post it. Otherwise, I will create a new issue to post it.

[ianlancetaylor]

I will reopen this issue because you requested it. But I want to be clear that I see no possibility of anything like this proposal being adopted for Go. I asked questions not because the answers affect the decision for this proposal, but as pointers for future work in a different direction.

The :reader syntax may not be the core of the proposal but it seems that it will start to appear in many places in Go code. That is also what I meant by the question about :reader as a type qualifier. We do not want to adopt a proposal that will lead people to scatter annotations across all Go code, and will then lead them to remove those annotations as code changes. This is the "const-poisoning" problem.

[zigo101]

@beoran
I will submit a new proposal which only support reader and read-only values. Final values will be removed.

@romshark
I found a flaw in the current proposal version (v9.1) in channel rules.

// By the current rule, we can't send values to (or receive values from) c.
final c = make(chan int, 1)

// p is deduced as a reader value
var p = &c

// c1 is deduced as a reader value.
// By the current rule, we can send values to (or receive values from) c1.
// It is broken!
var c1 = *p

In the next version (v9.a), the final value concept will be removed, also the related channel rules.

@ianlancetaylor

As some flaws are found in this current version (v9.1), and the difference between v9.1 and the next version v9.a is large. I will close this proposal and make a new one.

[zigo101]

The new proposal is here: #32245

[zigo101]

@ianlancetaylor

Abut read-only-poisoning, I think it is really a problem, but it is not a serious problem.
There are more benefits than drawbacks of read-only values.
There have already been many such poisoning alike situations,
for example, when the name of a parameter type is changed,
all the corresponding parameter type names in called functions must be also changed.

Read-only memory zone has negative impacts on run-time performance.