My Takeaways from Google Cloud Next23
Whether you’re deep in the tech game or just like to stay in the loop, I’ve got some juicy updates for you. Recently, I had the privilege of attending the Google Cloud Next ‘23 event in the iconic city of San Francisco.
Besides being blown away by San Francisco’s charm (a first-time visit for me!), the event was an absolute treasure trove of technological insights.
Now, while I took a plethora of notes, I’ve decided to share only the ones most relevant to our current infrastructure and those we believe could directly impact our organization at Datavalet. Trust me, it’s the good stuff!
A special thanks to our Enterprise Sales Director at Google Cloud, for extending an invite to both the lead of our DataOps team and myself. With her generous invitation, we were front and center for all the action. So, are you ready to delve into the key takeaways from Google Cloud Next with me ?
Unraveling AI and Cloud Innovations:
AI Platforms and Tools
1. Duet AI:
Google Cloud’s latest wonder, Duet AI, now in preview.
In our organization, we’ve been leveraging the power of GKE and BigQuery for quite some time. A few months back, we also implemented the Security Command Center Premium, and we’ve been dabbling with Looker Studio. So, imagine my excitement when I discovered that Duet AI is now in preview and integrating with all these products we actively use! This promises a boost in our current setup and some innovative workflows in the near future.
2. Vertex AI:
Our DataOps team, especially our dedicated ML engineer, frequently taps into the capabilities of Vertex AI for daily operations. So, the updates announced for this platform particularly caught my eye.
Vertex AI, known for its dedication to building generative AI apps, ML development, and MLOps, unveiled intriguing new features.
Among the highlights are updates to the PaLM 2, Codey, and imagen models. And an exciting addition: Colab Enterprise is now set to supercharge Vertex AI!
3. Vertex AI Search and Conversation:
Vertex AI Search and Conversation was a genuine revelation for me during the event. Formerly known as Enterprise Search on Generative AI App Builder and Conversational AI on Generative AI App Builder, these tools have now transitioned to GA (general availability).
Their emergence promises a transformation in the approach to constructing generative chatbots and custom search engines.
And, on a note of personal brainstorming, I’m playing with the idea of crafting an internal search engine to streamline knowledge sharing and the onboarding process for our development team.
Data Analytics
1. BigQuery:
BigQuery is recalibrating for the AI era, unveiling a variety of intriguing features. The most striking for me has been the introduction of BigQuery Studio. Beyond its consolidated workspace for data engineering, analytics, and predictive analysis, underscoring collaboration and seamless integration with tools such as Duet AI and Vertex AI foundation models, BigQuery Studio caught my interest because it aims to extend software development best practices to data assets. This includes the implementation of CI/CD, maintaining version history, and integrating source control, all of which enhance collaboration and data management within teams.
Delving deeper, here are some thoughts on how these integrations could reshape our infrastructure at Datavalet:
- Simplification: The fusion of BigQuery and Vertex AI foundation models is poised to dramatically simplify the process of analyzing unstructured data directly within BigQuery.
- Efficiency: It could potentially obviate the need for us to pen and manage custom Python code when invoking AI models.
- Streamlining Operations: The enhancements should also reduce the necessity of building and overseeing data pipelines between BigQuery and generative AI model APIs, leading to a more streamlined data management process.
2. Dataplex
Dataplex is receiving a noteworthy enhancement in its governance capabilities, with a special emphasis on data lineage and quality. Given our organization’s continuous efforts to comply with regulations and address data governance challenges, I am particularly excited about these updates and are looking forward to exploring these new governance capabilities in-depth in my technical roadmap.
Business Intelligence
With the integration of Duet AI in Looker, we can now have more intuitive conversations with our data, crafting comprehensive reports or intricate visualizations with mere sentences.
Modern Container-based Workloads
1. GKE Enterprise:
The unveiling of the new multi-cluster feature for GKE was exhilarating. GKE Enterprise introduces the concept of managing “fleets of clusters,” enabling each development team to move at a brisk pace, as they can operate independently. Diving into the history, back in 2019, Google Cloud rolled out Anthos, an innovative container platform designed to facilitate the seamless movement of workloads across different cloud platforms. The new GKE Enterprise amalgamates the prowess of Anthos with GKE, assisting enterprises that run multiple clusters in adeptly managing those intricate workloads. With the integration of Duet AI into both GKE and Cloud Run, Kubernetes operation is primed to become even more intuitive. It’s worth noting, however, that while this sounds promising, a comprehensive judgment would have to wait since GKE Enterprise will only be entering its preview phase this September.
2. Cloud Storage:
While Cloud Storage FUSE (Linux’s Filesystem in Userspace) was previously available in its open-source form, enabling objects in Cloud Storage buckets to be perceived and accessed like files on a local file system, recent advancements take this even further:
- The new
Cloud Storage FUSECSI driver for Google Kubernetes Engine (GKE) facilitates applications to mount Cloud Storage through the well-acquainted Kubernetes API. - Moreover, the
Cloud Storage FUSEfor AI/ML promises efficient file access, bringing a transformative change in how we run AI/ML workloads.
Developer Experience:
Google Cloud Jump Start Solutions (GA):
Introduced last April, I hadn’t gotten the chance to explore Google Cloud Jump Start Solutions in detail. Now that it’s generally available, solutions such as “Deploy an AI/ML Image Processing Pipeline” and “Build a Data Warehouse with BigQuery” have caught my attention. This will be particularly useful for onboarding my colleagues unfamiliar with Google Cloud, AI/ML or Data Engineering. And for us to quickly dive into a Google Cloud product.
Application Integration (GA):
- The recent unveiling of Application Integration introduces a persuasive no-code integration platform as a service (iPaaS).
- With the incorporation of
Duet AIinto Application Integration, I imagine the potential for crafting dynamic applications that could seamlessly integrate with an array of tools we currently leverage, such as JIRA, Confluence, ServiceNow, among others.
DevOps & CI/CD - GitLab & Google Cloud Partnership:
Great news for DevOps enthusiasts! My DevOps team is particularly thrilled.
GitLab & Google Cloud have announced a partnership. This alliance promises to seamlessly blend:
- GitLab’s strengths in source code management, planning, and CI/CD workflow.
- Advanced security and compliance features.
- Google’s Cloud console and Artifact Registry.
Given that we have already adopted GitLab and GitLab-ci as our principal DevOps tool for CI/CD, this collaboration spells exciting prospects for our future endeavors.
Security Posture
Security has always been an intricate and, frankly, intimidating subject for many, including me. The mere thought of vulnerabilities can be quite unsettling, especially from a software developer’s perspective. I took notes of some innovations that might make these concerns a bit more manageable.
- AI-powered risk summaries in the Security Command Center were a highlight. These summaries aim to convert complex attack graphs into easily comprehensible explanations of potential attack exposures, detailing affected assets and suggesting mitigation strategies. This initiative could be a game-changer for non-security experts like myself.
- Another intriguing feature is the introduction of attack path simulations. By integrating automated simulations with attack path analysis, the Security Command Center can mimic how real-world adversaries might target cloud resources. Such a practical and hands-on approach to understanding threats feels invaluable.
Overall, these advancements seem to offer clearer insights and actionable measures to fortify one’s security landscape, even for those of us who might not be experts in the field.
Next Steps: My Immediate TODOs Post-Event
As we wind down the year, the Google Cloud Next ‘23 event has provided an abundance of insights and opportunities to explore. From all the things I learned at the event, I’ve picked out six key actions I want to complete before the year ends:
Here’s my action plan inspired by these notes:
1. Take the AI Readiness Quick Check: Complete a quick assessment to understand Datavalet’s AI capabilities across six pillars.
2. Enroll for Duet AI and BigQuery Studio : To gain firsthand experience with its features, especially how it extends software development best practices to data assets.
3. Explore with my teams Duet AI’s Potential in Our Stack: Given we’re already using BigQuery, Looker, GKE, and the Security Command Center, assess how Duet AI could further augment our operations.
4. Engage with the DataOps Team on Vertex AI’s Latest Features: Ensure the team is updated and consider setting up training sessions or workshops.
5. Gather Feedback from the DevOps Team on GitLab & Google Cloud Partnership: Understand their perspective and insights on the new collaboration and how it can be integrated into current processes
6. Evaluate the Newly Announced Governance Capabilities of Dataplex: Focus on compliance and data governance and consider how this can be integrated into your organization’s data strategy.
In conclusion, whether you’re an executive leader, developer, architect, DevOps specialist, AI enthusiast, or hold any other role, I’d love to hear from you. If you attended the event or have insights from similar ones, what’s on your action plan? Let’s share our experiences and learn from each other!