Change LDAP - PANIC: runtime error: invalid memory address or nil pointer dereference

[FLeven]

• Gitea version (or commit ref): v1.14.3 or v1.13.7

• Git version: 2.32.0

• Operating system: Windows Server Core 2019

  • Git-go Version
  • gitea-1.14.3-windows-4.0-amd64
  • Docker and / or local

• Database (use [x]):

  • PostgreSQL
  • MySQL
  • MSSQL
  • SQLite

• Can you reproduce the bug at https://try.gitea.io:

  • Yes (provide example URL)
  • No

• Log gist (local):

2021/06/25 20:23:10 ...l/manager_windows.go:85:start() [T] Not running a service ... using the debug SVC manager
2021/06/25 20:23:10 ...l/manager_windows.go:101:Execute() [T] Awaiting server start-up
2021/06/25 20:23:10 cmd/web.go:82:runWeb() [I] Starting Gitea on PID: 5508
2021/06/25 20:23:10 ...dules/setting/git.go:101:newGit() [I] Git Version: 2.32.0, Wire Protocol Version 2 Enabled
2021/06/25 20:23:10 cmd/web.go:126:runWeb() [I] Global init
2021/06/25 20:23:10 ...dules/setting/git.go:101:newGit() [I] Git Version: 2.32.0, Wire Protocol Version 2 Enabled
2021/06/25 20:23:11 routers/init.go:134:GlobalInit() [T] AppPath: G:/Gitea/gitea-amd64.exe
2021/06/25 20:23:11 routers/init.go:135:GlobalInit() [T] AppWorkPath: G:/Gitea
2021/06/25 20:23:11 routers/init.go:136:GlobalInit() [T] Custom path: G:/Gitea/custom
2021/06/25 20:23:11 routers/init.go:137:GlobalInit() [T] Log path: G:/data/log
2021/06/25 20:23:11 routers/init.go:49:checkRunMode() [I] Run Mode: Prod
2021/06/25 20:23:12 ...dules/setting/log.go:287:newLogService() [I] Gitea v1.14.3 built with GNU Make 4.1, go1.16.5 : bindata, sqlite, sqlite_unlock_notify
2021/06/25 20:25:46 ...uters/routes/base.go:153:1() [E] PANIC: runtime error: invalid memory address or nil pointer dereference
/usr/local/go/src/runtime/panic.go:212 (0x1003864)
/usr/local/go/src/runtime/signal_windows.go:239 (0x100371e)
/source/models/login_source.go:262 (0x2c7981a)
/source/routers/admin/auths.go:309 (0x2c7982b)
/source/modules/web/route.go:64 (0x2ba1ffb)
/usr/local/go/src/net/http/server.go:2069 (0x13858aa)
/source/vendor/github.com/go-chi/chi/mux.go:436 (0x274ba6a)
/usr/local/go/src/net/http/server.go:2069 (0x13858aa)
/source/modules/web/route.go:103 (0x2ba26e9)
/usr/local/go/src/net/http/server.go:2069 (0x13858aa)
/source/modules/web/route.go:103 (0x2ba26e9)
/usr/local/go/src/net/http/server.go:2069 (0x13858aa)
/source/modules/web/route.go:103 (0x2ba26e9)
/usr/local/go/src/net/http/server.go:2069 (0x13858aa)
/source/modules/web/route.go:103 (0x2ba26e9)
/usr/local/go/src/net/http/server.go:2069 (0x13858aa)
/source/vendor/github.com/go-chi/chi/middleware/get_head.go:37 (0x2dd3fe1)
/usr/local/go/src/net/http/server.go:2069 (0x13858aa)
/source/modules/context/context.go:704 (0x276b121)
/usr/local/go/src/net/http/server.go:2069 (0x13858aa)
/source/routers/routes/base.go:94 (0x2ddb6ac)
/usr/local/go/src/net/http/server.go:2069 (0x13858aa)
/source/routers/routes/base.go:94 (0x2ddb6ac)
/usr/local/go/src/net/http/server.go:2069 (0x13858aa)
/source/modules/public/public.go:86 (0x1fd800c)
/usr/local/go/src/net/http/server.go:2069 (0x13858aa)
/source/modules/public/public.go:86 (0x1fd800c)
/usr/local/go/src/net/http/server.go:2069 (0x13858aa)
/source/routers/routes/base.go:199 (0x2ddd317)
/usr/local/go/src/net/http/server.go:2069 (0x13858aa)
/source/vendor/gitea.com/go-chi/session/session.go:256 (0x212ed6e)
/usr/local/go/src/net/http/server.go:2069 (0x13858aa)
/source/vendor/github.com/go-chi/chi/mux.go:70 (0x274956a)
/source/vendor/github.com/go-chi/chi/mux.go:311 (0x274fa1b)
/usr/local/go/src/net/http/server.go:2069 (0x13858aa)
/source/vendor/github.com/go-chi/chi/mux.go:436 (0x274ba6a)
/usr/local/go/src/net/http/server.go:2069 (0x13858aa)
/source/routers/routes/web.go:107 (0x2dde7a4)
/usr/local/go/src/net/http/server.go:2069 (0x13858aa)
/source/vendor/github.com/go-chi/chi/middleware/strip.go:30 (0x2dd4927)
/usr/local/go/src/net/http/server.go:2069 (0x13858aa)
/source/vendor/github.com/chi-middleware/proxy/middleware.go:37 (0x2dcfe53)
/usr/local/go/src/net/http/server.go:2069 (0x13858aa)
/source/routers/routes/web.go:63 (0x2dde2e7)
/usr/local/go/src/net/http/server.go:2069 (0x13858aa)
/source/vendor/github.com/go-chi/chi/mux.go:87 (0x27492f0)
/source/modules/web/route.go:298 (0x2ba151a)
/source/vendor/github.com/gorilla/context/context.go:141 (0x1d980fa)
/usr/local/go/src/net/http/server.go:2069 (0x13858aa)
/usr/local/go/src/net/http/server.go:2887 (0x1388ee9)
/usr/local/go/src/net/http/server.go:1952 (0x138436c)
/usr/local/go/src/runtime/asm_amd64.s:1371 (0x1026640)

Description

When I try to change any authentication source I get a 500 site and the above mentionesd runtime error and it is impossible to change any LDAP settings afterwards.

[zeripath]

The NPE is coming from looking at the login_source which is defined as a LDAP source but doesn't appear to have a Cfg which is an LDAP source.

There is very likely to be an error earlier that we are missing but it would be also helpful to review the database for this login_source looking at the login_sources with type 2 or 5 in particular looking for those with empty cfg

[FLeven]

Another Test on Win10, local SQL Server, local GiTEA same Problem.

I dont think it maters what you enter as authentication source.

DB output

id type name is_actived is_sync_enabled cfg created_unix updated_unix
1 2 test 1 1 ≻慎敭㨢琢獥≴∬潈瑳㨢氢椭獴搮≥∬潐瑲㨢㌶ⰶ匢捥牵瑩偹潲潴潣≬ㄺ∬歓灩敖楲祦㨢牴敵∬楂摮乄㨢∢∬楂摮慐獳潷摲㨢∢∬獕牥慂敳㨢䌢㵎獵牥ⱳ䍄氽椭獴䐬㵃敤Ⱒ唢敳䑲≎∺Ⱒ䄢瑴楲畢整獕牥慮敭㨢∢∬瑁牴扩瑵乥浡≥∺Ⱒ䄢瑴楲畢整畓湲浡≥∺Ⱒ䄢瑴楲畢整慍汩㨢洢楡≬∬瑁牴扩瑵獥湉楂摮㨢慦獬ⱥ䄢瑴楲畢整卓偈扵楬䭣祥㨢∢∬敓牡档慐敧楓敺㨢ⰰ䘢汩整≲∺挨㵮猥∩∬摁業䙮汩整≲∺Ⱒ刢獥牴捩整䙤汩整≲∺Ⱒ䔢慮汢摥㨢牴敵∬汁潬䑷慥瑣癩瑡䅥汬㨢慦獬ⱥ䜢潲灵䕳慮汢摥㨢慦獬ⱥ䜢潲灵乄㨢∢∬片畯䙰汩整≲∺Ⱒ䜢潲灵敍扭牥䥕≄∺Ⱒ唢敳啲䑉㨢∢} 1624653406 1624653426

CFG is not empty and this is a fresh instance.

[FLeven]

Going all the way to 1.12.6 and then it works again.

[zeripath]

The cfg entry is supposed to be json - that doesn't look like json...

[zeripath]

This is very odd, looking at the code

/source/routers/admin/auths.go:302:

	source, err := models.GetLoginSourceByID(ctx.ParamsInt64(":authid"))
	if err != nil {
		ctx.ServerError("GetLoginSourceByID", err)
		return
	}
	ctx.Data["Source"] = source
	ctx.Data["HasTLS"] = source.HasTLS()

// HasTLS returns true of this source supports TLS.
func (source *LoginSource) HasTLS() bool {
	return ((source.IsLDAP() || source.IsDLDAP()) &&
		source.LDAP().SecurityProtocol > ldap.SecurityProtocolUnencrypted) || // <- This is where the NPE occurs
		source.IsSMTP()
}

Now models.GetLoginSourceByID(ctx.ParamsInt64(":authid")) cannot return a nil otherwise it would error out.

So the only thing I can see is that the internal *ldap.Source in source.LDAP() is nil because:

// FromDB fills up a LDAPConfig from serialized format.
func (cfg *LDAPConfig) FromDB(bs []byte) error {
	json := jsoniter.ConfigCompatibleWithStandardLibrary
	return json.Unmarshal(bs, &cfg)
}

this fails or returns an error. But I don't understand that.

---

I can't reproduce this error here on SQLite or Postgres. So I think unless you can see something wrong with the json that is in that cfg, the only solution is going to be compiling in additional logging. Are you able to do that?

[FLeven]

No sry,
I pushed the working version 1.12.6 through all other version to find out where it will break, but it didn't, it is even working with the newest version. I checked the DB with every version and the CFG was readable.

Also the other authentication methods are also not working, they don't show a 500 error, but not all settings are saved.

id type name is_actived is_sync_enabled cfg created_unix updated_unix
1 2 test 1 1 ≻慎敭㨢琢獥≴∬潈瑳㨢氢椭獴搮≥∬潐瑲㨢㌶ⰶ匢捥牵瑩偹潲潴潣≬ㄺ∬歓灩敖楲祦㨢牴敵∬楂摮乄㨢∢∬楂摮慐獳潷摲㨢∢∬獕牥慂敳㨢䌢㵎獵牥ⱳ䍄氽椭獴䐬㵃敤Ⱒ唢敳䑲≎∺Ⱒ䄢瑴楲畢整獕牥慮敭㨢∢∬瑁牴扩瑵乥浡≥∺Ⱒ䄢瑴楲畢整畓湲浡≥∺Ⱒ䄢瑴楲畢整慍汩㨢洢楡≬∬瑁牴扩瑵獥湉楂摮㨢牴敵∬瑁牴扩瑵卥䡓畐汢捩敋≹∺Ⱒ匢慥捲偨条卥穩≥〺∬楆瑬牥㨢⠢湣┽⥳Ⱒ䄢浤湩楆瑬牥㨢∢∬敒瑳楲瑣摥楆瑬牥㨢∢∬湅扡敬≤琺畲ⱥ䄢汬睯敄捡楴慶整汁≬昺污敳∬片畯獰湅扡敬≤昺污敳∬片畯䑰≎∺Ⱒ䜢潲灵楆瑬牥㨢∢∬片畯䵰浥敢啲䑉㨢∢∬獕牥䥕≄∺索 1624653406 1624655344
2 2 ldap 1 1 ≻慎敭㨢氢慤≰∬潈瑳㨢氢慤⹰⵬瑩⹳敤Ⱒ倢牯≴㘺㘳∬敓畣楲祴牐瑯捯汯㨢ⰱ匢楫噰牥晩≹琺畲ⱥ䈢湩䑤≎∺乃琽獥ⱴ乃甽敳獲䐬㵃⵬瑩ⱳ䍄搽≥∬楂摮慐獳潷摲㨢␢牴扵ㅢ㌲Ⱒ唢敳䉲獡≥∺乃甽敳獲䐬㵃⵬瑩ⱳ䍄搽≥∬獕牥乄㨢∢∬瑁牴扩瑵啥敳湲浡≥∺Ⱒ䄢瑴楲畢整慎敭㨢∢∬瑁牴扩瑵卥牵慮敭㨢∢∬瑁牴扩瑵䵥楡≬∺慭汩Ⱒ䄢瑴楲畢整䥳䉮湩≤昺污敳∬瑁牴扩瑵卥䡓畐汢捩敋≹∺Ⱒ匢慥捲偨条卥穩≥〺∬楆瑬牥㨢⠢湣┽⥳Ⱒ䄢浤湩楆瑬牥㨢∢∬敒瑳楲瑣摥楆瑬牥㨢∢∬湅扡敬≤琺畲ⱥ䄢汬睯敄捡楴慶整汁≬昺污敳∬片畯獰湅扡敬≤昺污敳∬片畯䑰≎∺Ⱒ䜢潲灵楆瑬牥㨢∢∬片畯䵰浥敢啲䑉㨢∢∬獕牥䥕≄∺索 1624659388 1624659404
3 6 oa 1 0 ≻牐癯摩牥㨢朢瑩慥Ⱒ䌢楬湥䥴≄∺ㄲㄳ㌲‱Ⱒ䌢楬湥却捥敲≴∺㈱ㄳ㌲㌲Ⱒ伢数䥮䍄湯敮瑣畁潴楄捳癯牥啹䱒㨢∢∬畃瑳浯剕䵌灡楰杮㨢≻畁桴剕≌∺瑨灴㩳⼯楧整⹡潣⽭潬楧⽮慯瑵⽨畡桴牯穩≥∬潔敫啮䱒㨢栢瑴獰⼺术瑩慥挮浯氯杯湩漯畡桴愯捣獥彳潴敫≮∬牐景汩啥䱒㨢栢瑴獰⼺术瑩慥挮浯愯楰瘯⼱獵牥Ⱒ䔢慭汩剕≌∺索∬捉湯剕≌∺索 1624659444 1624659491
4 7 auth 1 0 ≻畁潴牃慥整獕牥≳琺畲ⱥ䄢瑵䅯瑣癩瑡啥敳獲㨢牴敵∬瑓楲䑰浯楡乮浡獥㨢牴敵∬敓慰慲潴割灥慬散敭瑮㨢ⴢⰢ䐢晥畡瑬慌杮慵敧㨢∢} 1624659517 1624659560

[zeripath]

Those cfg fields are being returned as bytes in whatever way you're pasting them in here. Likely this is because of our schema but there should be someway of casting them to strings for us to check if they're actually reasonable JSON.

[FLeven]

This is exactly how the config looks like in MS Sql Studio, just copy/pasted it here,

To be clear, the initial setup of any authentication source works and it look fine in the DB. LDAP for example works, login ok.
Only if you try to change anything and save the changes, then it gets scrambled in the DB and LDAP authentication is broken, no login possible. I have to add a new source and change every existing user to the new source, delete the old one and it is working again.

Workaround:
Install 1.12.6 and update to 1.13.7 (maybe you can skip this step), then update to 1.14.3 and everything works as expected.

[zeripath]

This is exactly how the config looks like in MS Sql Studio, just copy/pasted it here,

Before or after attempting to edit it? Would you be able to cast it as a string or even just give me the bytes themselves?

To be clear, the initial setup of any authentication source works and it look fine in the DB. LDAP for example works, login ok.
Only if you try to change anything and save the changes, then it gets scrambled in the DB and LDAP authentication is broken, no login possible. I have to add a new source and change every existing user to the new source, delete the old one and it is working again.

This suggests an issue with collations or encodings but I don't understand it.

Workaround:
Install 1.12.6 and update to 1.13.7 (maybe you can skip this step), then update to 1.14.3 and everything works as expected.

This isn't really a viable solution for most people - and it doesn't explain why 1.12.6 works when 1.14.3 doesn't.

[FLeven]

Plain export of the table:

working.txt
notworking.txt

I also setup the DB with a different collation (closer to UTF8mb4), but it changed nothing. Default language of my server/DB is german.

[zeripath]

Yep. OK looking at those bytes - they have been changed in to UTF16 with incorrectly prefixed with the UTF16 BOM.

[zeripath]

and I've managed to replicate the issue too.

[zeripath]

OK it's a bug with xorm.

[zeripath]

https://gitea.com/xorm/xorm/pulls/1957 is the bug fix PR.

The issue is that there is a fundamental mismatch with how Blob types and non-Blob types are handled for converts like this in Select, Insert and Update - and MSSQL unfortunately requires the special cast.

[zeripath]

There's probably another similar issue regarding repo units which are likely to suffer the same problem