Teleport 8.0 Test Plan #8665
Comments
|
Desktop Access: Seeing the arrow keys map incorrectly on macOS client. Same behavior in Chrome, Safari, and Firefox. This is using the built-in MacBook keyboard.
This is addressed with #8791 which still needs to be backported (#8813). |
|
Potential K8s access issue when used via TLS routing. I get: I'm using |
|
Desktop Access: disconnects due to client idle timeout fail to emit a disconnect event to the audit log (but otherwise disconnect correctly and emit a desktop session end event correctly). This is due to attempting to use the desktop ID as the event's "server ID." The server ID must be the Windows Desktop Service's HostUUID, otherwise the Fixed in #8828, which needs merge + backport. |
etcd Load Tests3 auth 2 proxy 1 node non-IoT3 auth 2 proxy 1 node IoT10k node non-IoT
Soak tests:10k nodes IoT
Soak tests:500 Trusted Clusters
|
We were attempting to use the desktop name as the server ID, but in order to publish audit events we must use the HostUUID of the windows_desktop_service. Updates #8665
|
Tested out several of the AWS deployment examples. Both Terraform examples worked fine with the 8.0.0-beta.2 AMIs (tested the simple example with the OSS AMIs, and the HA example with Enterprise). The CloudFormation example seems very broken. I'm told it hasn't been maintained in quite a while and am not sure if we care to validate it here. Some of its issues:
|
We were attempting to use the desktop name as the server ID, but in order to publish audit events we must use the HostUUID of the windows_desktop_service. Updates #8665
We were attempting to use the desktop name as the server ID, but in order to publish audit events we must use the HostUUID of the windows_desktop_service. Updates #8665
|
Desktop Access: Investigate required libraries. Issue #8765 |
|
Just filed #8860 for my issues with the CloudFormation example, though given #8665 (comment), I wonder if I hit a similar issue. |
|
Aggregate last 3 releases.
|
|
Re: Check agent forwarding is correct based on role and proxy mode.
Caveats: I'm still not sure what Proxy Mode means in this context, but I've interpreted to to mean the Fully aware that I may have been testing the wrong thing, I considered something that looks like this to be "allowed":
..and something that looks like this was considered "disallowed":
|
|
ALPN Proxy + Reverse Tunnel fails when ACME is used: Issue #8665 (comment) |
We were attempting to use the desktop name as the server ID, but in order to publish audit events we must use the HostUUID of the windows_desktop_service. Updates #8665
We were attempting to use the desktop name as the server ID, but in order to publish audit events we must use the HostUUID of the windows_desktop_service. Updates #8665
Dynamo (IoT)10k Scaling
SoakDynamo (non-IoT)10k Scaling
SoakDynamo (500 Trusted Cluster)
Notes
|
We were attempting to use the desktop name as the server ID, but in order to publish audit events we must use the HostUUID of the windows_desktop_service. Updates #8665
We were attempting to use the desktop name as the server ID, but in order to publish audit events we must use the HostUUID of the windows_desktop_service. Updates #8665
Manual Testing Plan
Below are the items that should be manually tested with each release of Teleport.
These tests should be run on both a fresh install of the version to be released
as well as an upgrade of the previous version of Teleport.
Adding nodes to a cluster @atburke
Labels @atburke
Trusted Clusters @atburke
RBAC @codingllama
Make sure that invalid and valid attempts are reflected in audit log.
Verify that custom PAM environment variables are available as expected. @xacrimon
Users @nklaassen
With every user combination, try to login and signup with invalid second factor, invalid password to see how the system reacts.
tsh mfa addtsh mfa addtsh mfa addtsh mfa lstsh mfa rmtsh mfa rmtsh mfa rmsecond_factor: oninauth_service, should failsecond_factor: optionalinauth_service, should succeedtsh mfa addBackends @Joerger
Session Recording @codingllama
Audit Log @quinqu
Failed login attempts are recorded
Interactive sessions have the correct Server ID
Node/Proxy ID may be found at
/var/lib/teleport/host_uuidin thecorresponding machine.
Node IDs may also be queried via
tctl nodes ls.Exec commands are recorded
scpcommands are recordedSubsystem results are recorded
Subsystem testing may be achieved using both
Recording Proxy mode
and
OpenSSH integration.
Assuming the proxy is
proxy.example.com:3023andnode1is a node runningOpenSSH/sshd, you may use the following command to trigger a subsystem audit
log:
sftp -o "ProxyCommand ssh -o 'ForwardAgent yes' -p 3023 %[email protected] -s proxy:%h:%p" root@node1Interact with a cluster using
tsh@tcscThese commands should ideally be tested for recording and non-recording modes as they are implemented in a different ways.
Interact with a cluster using
ssh@quinquMake sure to test both recording and regular proxy modes.
Interact with a cluster using the Web UI @tcsc
User accounting @xacrimon
/var/run/utmpon Linux./var/log/wtmpon Linux.Combinations @tcsc
For some manual testing, many combinations need to be tested. For example, for
interactive sessions the 12 combinations are below.
Teleport with EKS/GKE @smallinsky
Teleport with multiple Kubernetes clusters @r0mant
Note: you can use GKE or EKS or minikube to run Kubernetes clusters.
Minikube is the only caveat - it's not reachable publicly so don't run a proxy there.
tsh login, check thattsh kube lshas your clusterkubectl get nodes,kubectl exec -it $SOME_POD -- shtsh login, check thattsh kube lshas your clusterkubectl get nodes,kubectl exec -it $SOME_POD -- shtsh login, check thattsh kube lshas your clusterkubectl get nodes,kubectl exec -it $SOME_POD -- shtsh login, check thattsh kube lshas both clusterstsh kube loginkubectl get nodes,kubectl exec -it $SOME_POD -- shon the new clustertsh login, check thattsh kube lshas all clustersnameandlabelsStep 2login value matching the rowsnamecolumnnameorlabelsin the search bar worksnamecolumTeleport with FIPS mode @russjones
ACME @Joerger
Migrations @r0mant @russjones
SSH should work for both main and old clusters
SSH should work
Command Templates
When interacting with a cluster, the following command templates are useful:
OpenSSH
Teleport
Teleport with SSO Providers @benarent
Teleport Plugins @Joerger
WEB UI @kimlisa @rudream @gzdunek
Main
For main, test with a role that has access to all resources.
Top Nav
Side Nav
>, and expand has iconvServers aka Nodes
Add Serverbutton renders dialogue set toAutomaticallyviewRegenerate Scriptregenerates token value in the bash commandManuallytab renders manual stepsAutomaticallytab renders bash commandApplications
Add Applicationbutton renders dialogueGenerate Script, bash command is renderedRegeneratebutton regenerates token value in bash commandDatabases
Add Databasebutton renders dialogue for manual instructions:Step 4changesStep 5commandsActive Sessions
Audit log
Session Endedevent icon, takes user to session playerdetailsbuttonUsers
Auth Connectors
Auth Connectors Card Icons
Roles
Managed Clusters
Help & Support
Access Requests
Creating Access Requests
allow-roles). This role allows you to see the Role screen and ssh into all nodes.allow-users). This role session expires in 4 minutes, allows you to see Users screen, and denies access to all nodes.default)defaultassignedallow-rolesandallow-usersare listedViewing & Approving/Denying Requests
Create a user with the role
reviewerthat allows you to review all requests, and delete them.defaultif thresholds weren't defined in role, or blank if not named)Assuming Approved Requests
allow-rolesallows you to see roles screen and ssh into nodesallow-roles, verify that assumingallow-usersallows you to see users screen, and denies access to nodesswitching backgoes back to your default static roledefault, and requests that are not expired and are approved are assumable againAccess Request Waiting Room
Strategy Reason
Create the following role:
request_promptsettingsend request, pending dialogue rendersStrategy Always
With the previous role you created from
Strategy Reason, changerequest_accesstoalways:Logoutand clicking goes back to the login screenStrategy Optional
With the previous role you created from
Strategy Reason, changerequest_accesstooptional:Terminal
Node List Tab
Session Tab
$ sudo apt-get install mc$ mcSession Player
Invite Form
Login Form
Multi-factor Authentication (mfa)
Create/modify
teleport.yamland set the following authentication settings underauth_serviceMFA invite, login, password reset, change password
second_factortype toonand verify that mfa is required (no optionnonein dropdown)MFA require auth
Go to
Account Settings>Two-Factor Devicesand register a new deviceUsing the same user as above:
MFA Management
second_factorset tooffdisables adding devicesCloud
Invite/Reset
Recovery Code Management
Recovery Flow: Add new mfa device
Recovery Flow: Change password
Recovery Email
RBAC
Create a role, with no
allow.rulesdefined:Add Server, Application, Databases, Kubernetesbutton in each respective viewNodes,Apps,Databases, andKubernetesare listed underoptionsbutton inManage ClustersNote: User has read/create access_request access to their own requests, despite resource settings
Add the following under
spec.allow.rulesto enable read access to the audit log:Audit LogandSession Recordingsis accessibleAdd the following to enable read access to recorded sessions
Add the following to enable read access to the roles
Add the following to enable read access to the auth connectors
Add the following to enable read access to users
Add the following to enable read access to trusted clusters
Performance/Soak Test @fspmarshall @rosstimothy
Using
tsh benchtool, perform the soak tests and benchmark tests on the following configurations:Cluster with 10K nodes in normal (non-IOT) node mode with ETCD
Cluster with 10K nodes in normal (non-IOT) mode with DynamoDB
Cluster with 1K IOT nodes with ETCD
Cluster with 1K IOT nodes with DynamoDB
Cluster with 500 trusted clusters with ETCD
Cluster with 500 trusted clusters with DynamoDB
Soak Tests
Run 4hour soak test with a mix of interactive/non-interactive sessions:
Observe prometheus metrics for goroutines, open files, RAM, CPU, Timers and make sure there are no leaks
Breaking load tests
Load system with tsh bench to the capacity and publish maximum numbers of concurrent sessions with interactive
and non interactive tsh bench loads.
Teleport with Cloud Providers
AWS @timothyb89
GCP @xacrimon
IBM @xacrimon
Application Access @r0mant @smallinsky
debug_app: trueworks.name.rootProxyPublicAddrand well aspublicAddr.name.rootProxyPublicAddr.app.session.startandapp.session.chunkevents are created in the Audit Log.app.session.chunkpoints to a 5 minute session archive with multipleapp.session.requestevents inside.tsh play <chunk-id>can fetch and print a session chunk archive.tsh app login.tsh awscommands.tctl create.tctl create -f.tctl rm.Add Applicationdialogue works (refresh app screen to see it registered)Database Access @r0mant @smallinsky
db.session.startis emitted when you connect.db.session.endis emitted when you disconnect.db.session.queryis emitted when you execute a SQL query.tsh db lsshows only databases matching role'sdb_labels.db_users.db_names.db.session.startis emitted when connection attempt is denied.db_names.db.session.queryis emitted when command fails due to permissions.tsh db connect.tctl create.tctl create -f.tctl rm.name,description,type, andlabelsStep 2login value matching the rowsnamecolumnlabelsDesktop Access @zmb3
windows_desktop_services to the same Teleport cluster and connect to desktops on different AD domainsabcdefghijklmnopqrstuvwxyz1234567890-=!@#$%^&*()_+[]\{}|;':",./<>?~`, backspace, return and moving a cursor around with arrow keys works as expected with all supported browsers. Update: all other keys should now work. A known bug is that F11 (show desktop shortcut) doesn't work if you're on MacOS, as the OS seems to capture it and doesn't forward it to the browser. A useful tool for testing which keys are being registered on a windows machine is: https://dennisbabkin.com/kbdkeyinfo/TLS Routing @r0mant
v2configuration starts only a single listener.multiplexmodeauth_service.proxy_listener_mode: "multiplex"web_proxy_addr == tunnel_addrtsh db connectworks through proxy running inmultiplexmodetsh db proxywith a GUI client.multiplexmodessh -o "ForwardAgent yes" -o "ProxyCommand tsh proxy ssh %r@%h:%p" [email protected]ssh -o "ForwardAgent yes" -o "ProxyCommand tsh proxy ssh --cluster=leaf-cluster %r@%h:%p" [email protected]tsh sshaccess through proxy running in multiplex modemultiplexmodeThe text was updated successfully, but these errors were encountered: