Governance Practices in OCI: Organizing and Discovering Your Resources
Using Tagging, Advanced Resource Query, and more !
Are you ready to take your cloud environment to the next level? If you're like me, the idea of establishing governance practices may not sound like the most thrilling task. But the truth is, it's absolutely vital for ensuring security, cost-effectiveness, efficiency, and accountability. Luckily, the Oracle Cloud offers both the tools and a cloud adoption framework that breaks down the key pillars of a winning governance model, making it easier than ever to get on the right track. In this blog post, our focus will be on Tagging and its correlation with Cost Analysis. We will discuss various approaches to gain a comprehensive understanding of your tenancy. Let's explore this topic together.
Governance and Tagging
Oracle Cloud Adoption Framework describes the different pillars of the general governance model : Organize, Observe and Monitor, and Govern (link). Tagging plays an important part in organizing/governing your resources and are very versatile as they can be used for automation, technical reporting, cost reporting and cost-center split, auditing, and for various business purposes. They can be used closely with an IAM governance model to control who can access or modify certain resources.
Source: Establish a foundational Oracle Cloud Infrastructure Governance Model, Copyright © 2022, Oracle
Organizing resources through tagging may seem like a simple solution, but it is important to take the time to properly understand tagging best practices before implementing them hastily.
My first advice is to look directly into what we call Defined Tags because their existential purpose is to help you organize your tags -which ultimately lead to better organize your resources-. Unlike Free-Form Tags, they can be included into IAM policies and can help you better monitor your costs.
Regarding cost management, there's also a trap that most people tend to fall into when they start with tagging, because OCI also has a Cost Tracking Tags concept.
Cost Tracking Tags are a subset of Defined Tags that are used only if you need to set a soft limit on your spending - which is done through budgets. Said differently, you don't need to use Cost Tracking Tags just to monitor costs, Defined Tags will do the job. Also, keep in mind that there is a limit to the number of cost-tracking tags, so it's best to use Defined Tags instead.
My second piece of advice is that once you've understood how it works and set up a tagging plan, it is wise not to wait too much before starting tagging. This is especially true if you're tagging for cost analysis purposes as the cost analysis does not apply retroactively.
There's a lot more that can be done with Defined Tags, notably, you can enforce rules on the tags, for example obliging people to tag certain resources, making them choose among a pre-defined list of tag values, automatically tag when the resource gets created and personalize it with variables. Additionally, you have the ability to specify who can create and apply tags.
Ways to understand what's in your tenancy & exploring tagged resources
Once people start creating a lot of resources in a tenancy and tagging them, it can become difficult to have a full picture of what has been created and tagged in their tenancy.
I would suggest to start simple, and if out-of-the-box tools are not enough, look for more advanced solutions.
Let's take the example of cost analysis combined with tags.
Both Cost & Usage report and Cost Analysis dashboard offer detailed insights into costs and are ideal for getting started. Typically, we begin by identifying the services that are incurring costs. Then, we delve deeper to determine the specific resources within those services that are causing the expenses.
In the example below, I'm using the pre-defined report called Cost by Service and Tag, and I filter by a specific tag -I can add as many filters as I want-, from there I group by OCID because I want to understand specifically which unique resource is generating costs over a time period. Each resource having its own OCID, it's easy to look for it in the search bar.
Suppose I'd have a lot more tagged resources, I would then go to Tab Actions and export the csv.
On the other hand Cost & Usage Reports have all the information you need in a csv format including tags and resource identifers (OCIDs) but they are generally split into 6-hour time-windows which may oblige us to read multiple reports.
Having a full picture of what's going on in your tenancy could be also a next step in your investigation.
It goes without saying that navigating the console, service by service, filtering on regions and compartments, and using the top-search bar are the easiest and faster way to getting started exploring your resources.
For people looking to have a dynamic view of the systems provisioned in OCI, ie monitor system healths, logs, capacity etc, Observability & Management and service-specifics dashboards are the place to go (not covered in this article).
For more static views, another under-utilized feature set is the advanced search : Advanced Resource Query. This querying tool is accessible from the tool bar and provides a powerful query language to inspect a tenancy :
Example of Advanced Resource Query
All query-able resources are here and the syntax of the query language is here. No panic here, no need to go on training to learn it, if you've ever run a basic SQL query in your life with a where clause, you'll manage it.
The interface combo box offers a few sample queries that you can customize directly. All the search results can be filtered and clicked into for more details.
What if you want to have a larger/more complete picture?
The same queries can be run on the OCI CLI.
oci search resource structured-search --query-text "query all resources" > output000.json
Which will produce a list of items in a json format :
{
"data": {
"items": [
{
"additional-details": {},
"availability-domain": null,
"compartment-id": "ocid1.compartment.oc1..aaaaaaaaa",
"defined-tags": {},
"display-name": "emdbhost1-scan21",
"freeform-tags": {},
"identifier": "ocid1.privateip.oc1.eu-frankfurt-1.aaaaaaaccc",
"identity-context": {},
"lifecycle-state": "AVAILABLE",
"resource-type": "PrivateIp",
"search-context": null,
"system-tags": {},
"time-created": "2023-05-03T09:44:49.019000+00:00"
},
{
"additional-details": {},
"availability-domain": null,
"compartment-id": "ocid1.compartment.oc1..aaaaaaaarnaaaa",
"defined-tags": {},
"display-name": "emdbhost2-test",
"freeform-tags": {},
"identifier": "ocid1.privateip.oc1.eu-frankfurt-1.dddddddd",
"identity-context": {},
"lifecycle-state": "AVAILABLE",
"resource-type": "PrivateIp",
"search-context": null,
"system-tags": {},
"time-created": "2023-05-03T09:44:38.582000+00:00"
},
...
In case the number of resources exceeds 1000 elements, you'll need to overcome the paging problem (here's a possible solution if you don't plan to do it yourself).
The beauty of this is that these queries can be modified to search for specific tags namespaces, keys, or values (please note this is from a windows cmd)
oci search resource structured-search --query-text "query all resources where definedTags.namespace ='WHCostNamespace' "
oci search resource structured-search --query-text "query all resources where (definedTags.namespace ='WHCostNamespace' && definedTags.value='WHTag3') " | jq ".data.items[] | ."
On the second one, notice that I installed jq tool -which is a json command line processor - on my local environment, this is interesting to overcome potential limitations of the query language.
For example, you could use the following command to query on all resources that have been tagged with a Defined Tag :
oci search resource structured-search --query-text "query all resources" | jq ".data.items[] | select(.["defined-tags"] | length > 1 ) | ."
At this stage you should be able to play with the query language and jq to answer any of your specific requirements.
Go beyond the tags !
For people who wish to go further, I'd suggest also to have a look at Show OCI toolkit. SHOWOCI is an open-source tool that promises to "extract the list of resources from your tenant. It covers most of OCI components, Authentication by User or Compute using instance principals, Output can be printer friendly, CSV files or JSON file."
The outputs are complete and better formatted than with the solution we've previously covered.
Please note that this is not an official Oracle product.
There are plenty of other ways to better understand what is happening in a tenancy, we can mention (I probably forgot others, feel free to drop me a note/comment ;-) ) :
Cloud Guard, a cloud-native service built specifically to assess the security state of a tenancy and offers both a static and dynamic view of it with recommendations on how to address detected issues.
Network Analyzer to visualize & understand how network entities are connected and how routing is configured.
Resource Manager specifically to automatically discover OCI resources and generate Terraform configuration/state files for it in order to re-provision those resources later or elsewhere (infra-as-code). Think of this in conjunction with the OCI Designer Toolkit if you're an architect willing to document your system or just into graphical representations of things.
In summary we've seen the importance of following governance best practices by using defined tags best practices and making use of a few tools that Oracle and its community provide.
I hope you enjoyed reading this article and that you'll be the king/queen of advanced queries soon and know completely what's inside your tenancy !
Disclaimer : Views are my own , none of the ideas expressed in this post are shared, supported, or endorsed in any manner by my current employer.
Photo by Mehmet Turgut Kirkgoz from Pexels: https://www.pexels.com/photo/colorful-geometric-figures-on-blue-background-16574831/