There are a few different ways to work with it all.
My personal preference would be to use familiar AWS service icons as describers.
There are also custom ones for a VPC, AWS Step Functions border, etc.
This post for Entity Relationship Diagrams
There are two different styles of ERDs.