1. 前言
企业在上云过程中,随着不同业务负载逐步迁移至云上,在设计云安全基线过程中,不仅需要设定企业整体范围的统一安全基线标准,同时也需要考虑到不同业务部门的特定安全需求。面对大量云业务负载跨多个Organizations、多个账户,如何快速有效执行日常安全基线检查,这对于大多数企业信息安全团队会是一个挑战。同时,在大量Account中进行资源统计和配置查询是非常繁琐的工作,通常会利用MSP和云管平台等来进行,但是对于没有接入上述平台的云端环境或需要进行快速、复杂查询的情况,就可以借助Steampipe Amazon Web Services plugin来简化操作了。本文我们将会介绍利用开源即时查询工具Steampipe在亚马逊云科技服务快速构建云端安全基线检查方案。
具体来说,本方案基于Steampipe配置方案,适用于多个Amazon Organizations,每个Organizations中有大量Account使用AssumeRole+MFA的方式以特定Account中的user对Organizations中的Account进行访问的环境。这种IAM和Account配置是企业AWS环境中常见的配置方式,其他账号配置方式可基于此配置模板进行修改匹配。
基于Steampipe引擎 可以使用SQL语句来对多个Organizations和大量Account的环境进行有效、复杂的资源统计和配置查询。
相对于很多沉重复杂的商用云管和安全产品,Steampipe具有不需要对account环境进行变更配置,轻量化部署可以便利的任何位置直接运行,PostgreSQL标准数据接口提供了更多扩展可能等优点,是“小而美“的云CMDB和安全基线检查方案,非常适合作为”敏捷运维“的基础工具栈使用。
2. 前置准备:
Steampipe可以部署在个人电脑终端wsl环境和Linux运维工作站中 也可以配合instance role部署在EC2实例中,以下以常用的WSL环境为例,其他Linux环境大同小异。配置Steampipe前需要确保以下内容就绪:
- 安装awscli,或者手工创建aws credentials和config文件。参考:
https://docs.thinkwithwp.com/cli/latest/userguide/getting-started-install.html
- 安装aws-vault。参考:https://github.com/99designs/aws-vault
- 安装最新版本的Steampipe。参考:https://Steampipe.io/downloads
- 安装aws plugin和其他所需plugin。参考https://hub.Steampipe.io/plugins/turbot/aws, (安装plugin如需要使用镜像库和基于文件的安装方式,参考
https://Steampipe.io/docs/using-Steampipe/managing-plugins )
- 准备好访问云端环境所需的credential(AKSK, MFA_serial, role_arn)信息,Account id list等基本环境信息,所使用的role应具备查询所需要的权限,如“ReadOnlyAccess”
3. 配置步骤
Organization环境下复杂OU和account结构的steampipe connection配置是非常繁琐枯燥的工作,笔者编写了一个自动化小工具帮助快速完成这一配置,可以从https://github.com/happy240/steampipe-conn-generator-for-aws-organization 获取。以下是详细的配置逻辑:
3.1 配置aws-vault credential profile
aws-vault add <profile name>
根据提示输入AKSK等信息。
说明: 可以使用aws-vault list查看已配置的profile列表,也可以在~/.aws/config中找到,并进一步修改profile的配置信息。
3.2 配置aws config文件增加MFA信息
在~/.aws/config文件,找到对应的profile增加MFA信息,示例配置以中国区域为例。 如果在Global区域配置,注意修改aws-cn为aws。
[profile <profile name>]
MFA_serial=arn:aws-cn:iam::<Account1 id>:MFA/<MFA id>
3.3 配置aws credentials文件引用aws-vault profile
在~/.aws/credentials文件中为每一个需要查询的AWS Account增加一条记录。
注意:
- profile名称可以直接使用Account id,也可以使用便于分辨的名称
- 保险起见,aws-vault路径应设置为绝对路径,而不是使用PATH环境变量,避免环境变量设置错误造成无法获取credential的情况。
- 对于中国区环境 aws-vault指令带有–region参数。
[Account1]
credential_process=aws-vault exec -j <profile name> --region=cn-north-1
[Account2]
source_profile=orgprefix_Account1
role_arn=arn:aws-cn:iam::<Account2 id>:role/<role id>
[Account3]
source_profile=orgprefix_Account1
role_arn=arn:aws-cn:iam::<Account3 id>:role/<role id>
3.4 配置Steampipe aws plugin connection文件
在~/.Steampipe/config/aws.spc文件中为每一个AWS Account增加一条记录。
注意:
- connection名称务必根据Organizations等分组信息增加统一前缀,方便后续配置aggregation。
- 在中国区域,regions参数不能使用*通配符, 否则会错误调用Global region的API。
- 可以为不同region设置不同的connection profile方便进行不同区域的查询或提高查询性能。
connection "orgprefix_Account1" {
plugin = "aws"
profile = "Account1"
regions = ["cn-north-1","cn-northwest-1"]
}
connection "orgprefix_Account2" {
plugin = "aws"
profile = "Account2"
regions = ["cn-north-1","cn-northwest-1"]
}
connection "orgprefix_Account3" {
plugin = "aws"
profile = "Account3"
regions = ["cn-north-1","cn-northwest-1"]
}
3.5 根据需要配置aggregation connection
在~/.Steampipe/config/aws.spc文件中,根据查询需要,将要合并至一起查询的account connection配置为aggregation。例如:
connection "orgprefix_all" {
type = "aggregator"
plugin = "aws"
connections = ["orgprefix_*"]
}
保存~/.Steampipe/config/aws.spc文件。
4. 查询步骤
4.1 输入MFA并缓存session信息
注意:Steampipe目前无法直接提示输入MFA,因此如果使用了MFA,应在查询前通过aws-vault提前获取并缓存session token,否则在未提前验证或session过期(默认session超时时间是1小时)的情况下,Steampipe查询会因为提示输入MFA卡死,需要重开shell。
aws-vault exec -j <profile name> --region cn-north-1
根据提示输入MFA信息。
注意: 如果需要较长时间进行查询操作 可以最长使用”-d 12h”参数执行aws-vault exec:
aws-vault exec -j <profile name> -d 12h --region cn-north-1
如需要提前刷新session token:
aws-vault rotate <profile name>
4.2 Steampipe进行查询
使用Steampipe查询,以下举例几个常用的资源查询语句:
# 启动交互式查
Steampipe query
# 统计每个Account中的ec2数
select Account_id,count(*) as countalb from orgprefix_all.aws_ec2_instance
group by Account_id
# 统计每个Account中的ALB数
select Account_id,count(*) as countalb from
orgprefix_all.aws_ec2_application_load_balancer group by Account_id
# 统计每个Account中的VPC数
select Account_id,count(*) as countalb from orgprefix_all.aws_vpc group by
Account_id
4.3 输出到csv
将查询结果输出到.csv文件。
Steampipe query "<sql query>" --output csv > <filename>.csv
4.4 可视化和合规扫描
根据需要获取Steampipe AWS Compliance Mod和AWS Insights Mod,参考:
https://hub.steampipe.io/mods/turbot/aws_compliance
https://hub.steampipe.io/mods/turbot/aws_insights
进入对应Mod目录,执行’steampipe dashboard’即可启动可视化面板
5. 查询示例
以下是一些场景示例设计及对应的查询语句,建议结合实际安全基线设定灵活组合使用。
示例1: 查询一组账户开通的特定资源数量
前置条件:已配置connection aggregator ‘org_all’,以下语句统计aggregator connection中每个account中的ALB数量。
select account_id,count(*) as countalb from
org_all.aws_ec2_application_load_balancer group by account_id
select
load_balancer_arns,count(*)
from
org_all.aws_ec2_target_group
cross join jsonb_array_elements(target_health_descriptions) as target
group by load_balancer_arns
select account_id,count(*) as counteip from account_all.aws_vpc_eip group by
account_id
select group_id,ip_permission -> 'FromPort' as OpenPort from
account_all.aws_vpc_security_group cross join
jsonb_array_elements(ip_permissions) as ip_permission where (ip_permission -
>> 'FromPort'='80' and ip_permission ->> 'ToPort'='80') or (ip_permission -
>> 'FromPort'='443' and ip_permission ->> 'ToPort'='443') or (ip_permission
->> 'FromPort'='8080' and ip_permission ->> 'ToPort'='8080') or
(ip_permission ->> 'FromPort'='8081' and ip_permission ->> 'ToPort'='8081')
or (ip_permission ->> 'FromPort'='3128' and ip_permission ->>
'ToPort'='3128') or (ip_permission ->> 'FromPort'='9080' and ip_permission -
>> 'ToPort'='9080')
示例2:查询开放web端口的EIP
select eip.*,sg ->> 'GroupId' as sgid,websg.OpenPort from (select
account_id,instance_id,network_interface_id,private_ip_address,public_ip
from account_all.aws_vpc_eip) as eip inner join (select
network_interface_id,groups from account_all.aws_ec2_network_interface) as
eni on eip.network_interface_id=eni.network_interface_id cross join
jsonb_array_elements(eni.groups) as sg inner join (select
group_id,ip_permission -> 'FromPort' as OpenPort from
account_all.aws_vpc_security_group cross join
jsonb_array_elements(ip_permissions) as ip_permission where (ip_permission -
>> 'FromPort'='80' and ip_permission ->> 'ToPort'='80') or (ip_permission -
>> 'FromPort'='443' and ip_permission ->> 'ToPort'='443') or (ip_permission
->> 'FromPort'='8080' and ip_permission ->> 'ToPort'='8080') or
(ip_permission ->> 'FromPort'='8081' and ip_permission ->> 'ToPort'='8081')
or (ip_permission ->> 'FromPort'='3128' and ip_permission ->>
'ToPort'='3128') or (ip_permission ->> 'FromPort'='9080' and ip_permission -
>> 'ToPort'='9080')) as websg on sg ->> 'GroupId'=websg.group_id
示例3:查询开放web端口的NLB
select nlb.account_id, nlb.name,nlb.arn from
account_all.aws_ec2_network_load_balancer as nlb inner join
(select load_balancer_arn,port from
account_all.aws_ec2_load_balancer_listener where port in
(80,443,8080,8081,3128,9080)) as listener on
nlb.arn=listener.load_balancer_arn
示例4:查询特定账户所使用Amazon OpenSearch版本
select account_id, arn, domain_id, domain_name, elasticsearch_version,
enabled, endpoint from account_all.aws_elasticsearch_domain
示例5:查询&过滤特定机型
select account_id, instance_id, instance_type, instance_state from
account_all.aws_ec2_instance where instance_type not like 'a1%' and
instance_type not like 'c5%' and instance_type not like 'c6%' and
instance_type not like 'd3%' and instance_type not like 'dl%' and
instance_type not like 'g4%' and instance_type not like 'g5%' and
instance_type not like 'hpc6%' and instance_type not like 'i3en%' and
instance_type not like 'im4gn%' and instance_type not like 'inf1%' and
instance_type not like 'is4gen%' and instance_type not like 'm5%' and
instance_type not like 'm6%' and instance_type not like 'p3dn%' and
instance_type not like 'p4%' and instance_type not like 'r5%' and
instance_type not like 'r6%' and instance_type not like 't3%' and
instance_type not like 't4%' and instance_type not like 'vt1%' and
instance_type not like 'x2gd%' and instance_type not like 'z1d%'
示例6:查询特定实例AMI信息
select instance.account_id,instance.instance_id,ami.*,instance.tags from
account_all.aws_ec2_instance as instance left outer join (select
account_id,image_id,name,platform_details,description from
account_all.aws_ec2_ami_shared where owner_id='141808717104') as ami on
instance.account_id=ami.account_id and instance.image_id=ami.image_id
示例7:查询特定实例SSM托管状态
select ec2.account_id as account_id, ec2.instance_id, mi.computer_name,
mi.platform_name, mi.platform_version, ec2.instance_type,
ec2.instance_state, mi.agent_version, mi.is_latest_version, mi.ping_status,
ec2.launch_time from (select
account_id,instance_id,instance_state,instance_type,launch_time from
account_all.aws_ec2_instance) as ec2 left outer join (select
account_id,agent_version,computer_name,instance_id,is_latest_version,ping_st
atus,platform_name,platform_version from
account_all.aws_ssm_managed_instance) as mi on
ec2.instance_id=mi.instance_id order by account_id
示例8:查询ENI与RDS关联情况
select i.*,r.* from (select rdsinstance.*,rdsinstanceip.ip from (select
account_id, db_instance_identifier, arn, endpoint_address, engine,
engine_version, vpc_id, subnets from account_all.aws_rds_db_instance) as
rdsinstance join (select domain,ip from net.net_dns_record where domain in
(select endpoint_address from account_all.aws_rds_db_instance) and type =
'A' and target is null) as rdsinstanceip on
rdsinstance.endpoint_address=rdsinstanceip.domain) as r join (select
account_id,network_interface_id,private_ip_address,association_public_ip
from account_all.aws_ec2_network_interface where
attached_instance_owner_id='amazon-rds') as i on r.account_id=i.account_id
and (r.ip=i.private_ip_address or r.ip=i.association_public_ip)
示例9:查询未使用的ENI
select
account_id,network_interface_id,description,private_ip_address,association_p
ublic_ip from account_all.aws_ec2_network_interface where status!='in-use'
6. 结尾
至此,我们介绍了一个云原生的轻量级安全基线检查方案,涉及该方案的前置准备、部署过程以及使用步骤。同时,附上基于Steampipe的常用安全基线检查查询语句示例,以供读者参考,进一步结合自身实际场景组合执行日常安全基线检查。
7. 参考资料
更多关于Steampipe connection和查询的信息,请参考:
https://thinkwithwp.com/cn/blogs/opensource/querying-aws-at-scale-across-apis-regions-and-accounts/
本篇作者