首页 > 其他分享 >【Azure Event Hub】Schema Registry 在China Azure门户上不能创建的替代方案

【Azure Event Hub】Schema Registry 在China Azure门户上不能创建的替代方案

时间:2024-04-28 18:58:54浏览次数:26  
标签:const string Hub China new Azure using event

问题描述

创建Event Hub服务后,标准版的定价层功能中有Schema Registry的功能,但是根据官方文档,在门户中确无法创建。

 

问题解答

什么是 Azure 架构注册表?
Azure 架构注册表是事件中心的一项功能,它为事件驱动的应用程序和以消息为中心的应用程序的架构提供一个中心存储库。

它使生成者和使用者应用程序可以灵活地交换数据,而无需管理和共享架构

它还为可重用架构提供了一个简单的治理框架,并通过分组构造(架构组)定义了架构之间的关系。

 使用 Apache Avro 等架构驱动的序列化框架,将序列化元数据移到共享架构还有助于降低每条消息的开销。 这是因为每条消息不需要像 JSON 等标记格式那样包含元数据 (类型信息和字段名称)。

将架构与事件一起存储,并将其存储在事件基础结构中,可确保序列化或反序列化所需的元数据始终是可访问的,并且架构始终置于正确的位置。

  由于在UI上无法创建Scheme Registry功能, 可以通过REST API 或者是Azure CLI命令:

# REST API

PUT https://management.chinacloudapi.cn/subscriptions/<>/resourceGroups/<>/providers/Microsoft.EventHub/namespaces/<>/schemagroups/<>?api-version=2021-11-01

{
  "properties": {
    "groupProperties": {},
    "schemaCompatibility": "Forward",
    "schemaType": "Avro"
  }
}

参考资料:https://learn.microsoft.com/en-us/rest/api/eventhub/schema-registry/create-or-update?view=rest-eventhub-2021-11-01&tabs=HTTP

 

# Azure Cli

az eventhubs namespace schema-registry create --name
                                              --namespace-name
                                              --resource-group
                                              [--group-properties]
                                              [--schema-compatibility {Backward, Forward, None}]
                                              [--schema-type {Avro, Unknown}]

 

参考资料:https://learn.microsoft.com/en-us/cli/azure/eventhubs/namespace/schema-registry?view=azure-cli-latest

 

# 使用示例

在使用事件中心 .NET SDK (AMQP) 流式处理事件时使用 Avro 架构验证 : https://learn.microsoft.com/zh-cn/azure/event-hubs/schema-registry-dotnet-send-receive-quickstart

发送消息

using Azure.Data.SchemaRegistry;
using Azure.Identity;
using Microsoft.Azure.Data.SchemaRegistry.ApacheAvro;
using Azure.Messaging.EventHubs;
using Azure.Messaging.EventHubs.Producer;

using Microsoft.Azure.Data.SchemaRegistry.example;

// connection string to the Event Hubs namespace
const string connectionString = "EVENTHUBSNAMESPACECONNECTIONSTRING";

// name of the event hub
const string eventHubName = "EVENTHUBNAME";

// Schema Registry endpoint 
const string schemaRegistryEndpoint = "EVENTHUBSNAMESPACENAME.servicebus.windows.net";

// name of the consumer group   
const string schemaGroup = "SCHEMAGROUPNAME";

// The Event Hubs client types are safe to cache and use as a singleton for the lifetime
// of the application, which is best practice when events are being published or read regularly.
EventHubProducerClient producerClient;

// Create a producer client that you can use to send events to an event hub
producerClient = new EventHubProducerClient(connectionString, eventHubName);

// Create a schema registry client that you can use to serialize and validate data.  
var schemaRegistryClient = new SchemaRegistryClient(schemaRegistryEndpoint, new DefaultAzureCredential());

// Create an Avro object serializer using the Schema Registry client object. 
var serializer = new SchemaRegistryAvroSerializer(schemaRegistryClient, schemaGroup, new SchemaRegistryAvroSerializerOptions { AutoRegisterSchemas = true });

// Create a new order object using the generated type/class 'Order'. 
var sampleOrder = new Order { id = "1234", amount = 45.29, description = "First sample order." };
EventData eventData = (EventData)await serializer.SerializeAsync(sampleOrder, messageType: typeof(EventData));

// Create a batch of events 
using EventDataBatch eventBatch = await producerClient.CreateBatchAsync();

// Add the event data to the event batch. 
eventBatch.TryAdd(eventData);

// Send the batch of events to the event hub. 
await producerClient.SendAsync(eventBatch);
Console.WriteLine("A batch of 1 order has been published.");       

 

消费消息

using Azure.Data.SchemaRegistry;
using Azure.Identity;
using Microsoft.Azure.Data.SchemaRegistry.ApacheAvro;
using Azure.Storage.Blobs;
using Azure.Messaging.EventHubs;
using Azure.Messaging.EventHubs.Consumer;
using Azure.Messaging.EventHubs.Processor;

using Microsoft.Azure.Data.SchemaRegistry.example;


// connection string to the Event Hubs namespace
const string connectionString = "EVENTHUBSNAMESPACECONNECTIONSTRING";

// name of the event hub
const string eventHubName = "EVENTHUBNAME";

// Schema Registry endpoint 
const string schemaRegistryEndpoint = "EVENTHUBSNAMESPACENAME.servicebus.windows.net";

// name of the consumer group   
const string schemaGroup = "SCHEMAGROUPNAME";

// connection string for the Azure Storage account
const string blobStorageConnectionString = "AZURESTORAGECONNECTIONSTRING";

// name of the blob container that will be userd as a checkpoint store
const string blobContainerName = "BLOBCONTAINERNAME";

// Create a blob container client that the event processor will use 
BlobContainerClient storageClient = new BlobContainerClient(blobStorageConnectionString, blobContainerName);

// Create an event processor client to process events in the event hub
EventProcessorClient processor = new EventProcessorClient(storageClient, EventHubConsumerClient.DefaultConsumerGroupName, connectionString, eventHubName);

// Register handlers for processing events and handling errors
processor.ProcessEventAsync += ProcessEventHandler;
processor.ProcessErrorAsync += ProcessErrorHandler;

// Start the processing
await processor.StartProcessingAsync();

// Wait for 30 seconds for the events to be processed
await Task.Delay(TimeSpan.FromSeconds(30));

// Stop the processing
await processor.StopProcessingAsync();

static async Task ProcessEventHandler(ProcessEventArgs eventArgs)
{
    // Create a schema registry client that you can use to serialize and validate data.  
    var schemaRegistryClient = new SchemaRegistryClient(schemaRegistryEndpoint, new DefaultAzureCredential());

    // Create an Avro object serializer using the Schema Registry client object. 
    var serializer = new SchemaRegistryAvroSerializer(schemaRegistryClient, schemaGroup, new SchemaRegistryAvroSerializerOptions { AutoRegisterSchemas = true });

    // Deserialized data in the received event using the schema 
    Order sampleOrder = (Order)await serializer.DeserializeAsync(eventArgs.Data, typeof(Order));

    // Print the received event
    Console.WriteLine($"Received order with ID: {sampleOrder.id}, amount: {sampleOrder.amount}, description: {sampleOrder.description}");

       await eventArgs.UpdateCheckpointAsync(eventArgs.CancellationToken);
    }

    static Task ProcessErrorHandler(ProcessErrorEventArgs eventArgs)
{
    // Write details about the error to the console window
    Console.WriteLine($"\tPartition '{eventArgs.PartitionId}': an unhandled exception was encountered. This was not expected to happen.");
    Console.WriteLine(eventArgs.Exception.Message);
    return Task.CompletedTask;
}      

 

    【END】

标签:const,string,Hub,China,new,Azure,using,event
From: https://www.cnblogs.com/lulight/p/18164293

相关文章

  • Github管理代码学习
    版本冲突github代码管理学习。关于版本控制,假如有两个人合作,合作者a修改了合作者b的代码,并在服务器中更新。而合作者b感觉更新的内容是错的,想基于未更新版本进行修改。这种情况该怎么解决在使用GitHub进行代码管理时,如果出现你描述的情况,合作者B可以采取以下几个步骤来解决问题:......
  • 2018-2019 ACM-ICPC, China Multi-Provincial Collegiate Programming Contest
    A.MaximumElementInAStack按照题目模拟就好,栈内的最大值可以维护单调栈解决。#include<bits/stdc++.h>usingnamespacestd;usingi64=longlong;usingui32=unsignedint;ui32SA,SB,SC;ui32rng61(){SA^=SA<<16;SA^=SA>>5;SA^......
  • 39、BlackRose(VulnHub)
    BlackRose一、nmap二、web渗透随便看看注册进去账号:xxxxxx密码:xxxxxx目录爆破有很多特殊的目录,不过访问后都重定向了burpsuite改包进admin查看xxxxxx用户数据包抓一些xxxxxx用户的一些记录包,看看有什么可用的signature=&command=id&indexcsrf=3cb58993bfd71f......
  • 记录VS Github Copilot入门上手
    我拉取了一个简单的开源项目,在此基础上测试Copilot功能。功能点盘点:1.根据要求和上下文的代码生成你想要的方法 2.代码写的过程中自动提示和补全代码此功能非常强大,会不停的帮你补全,分析生成你可能写入的代码,它会结合你的项目中的代码帮你生成。基本上只用写好备注,等待生......
  • 图文解说ChinaCock日期组件CCDateTimePicker(二)
    上文,介绍了CCDateTimePicker基本用法,实现日期、时间等各种日期格式的输入,用法简单,代码简洁。能不能用这个控件,来实现自定义的输入格式呢?答案是能。比如:我的需要求就遇到这样的情况,用户要选择星期几的方式,如下图,是已经实现的结果: 接下来看看如何实现的?第一步,先定义一个数组: ......
  • Windows git绑定GitHub
    1、打开gitBash2、输入ssh-keygen-trsa-C 'GitHub邮箱账号'  // 一直回车3、获取生成秘钥id_rsa.pub从C盘-用户-用户名目录对应的.ssh文件,打开id_rsa.pub文件,复制里面的数据4、将复制的秘钥添加到github上步骤1:点击头像打开settings步骤2:打开页面左边的SSHan......
  • Azure Storage (30) 基于属性的访问控制(ABAC)
    《WindowsAzurePlatform系列文章目录》 我们在使用AzureStorage的时候,经常会基于属性来进行访问控制,我们假设一个场景1:(1)用户:productionuser01可以访问到container以production开头的文件内容,如contoso.blob.core.windows.net/production01contoso.blob.c......
  • 支持国际学术资源开放(版权费用 Open Access),支持SCI-HUB,向Sci-hub致敬
    在去年多次向中国红字会捐款后再次决定向公益事业捐款,这次的捐款对象是SCI-HUB,可以说这是我们这种弱势的无大单位庇佑的散researcher的必备工具,多年来一直在使用,这次突然看到有支付宝捐款的途径,于是便决定捐款以作支持,虽然个人能力有限难以有较大额度的支持,但是也是小额支持下,聊表......
  • 《HelloGitHub》第 97 期
    兴趣是最好的老师,HelloGitHub让你对编程感兴趣!简介HelloGitHub分享GitHub上有趣、入门级的开源项目。https://github.com/521xueweihan/HelloGitHub这里有实战项目、入门教程、黑科技、开源书籍、大厂开源项目等,涵盖多种编程语言Python、Java、Go、C/C++、Swift.........
  • Jenkins 配置 以接收 GitHub 的 WebHook 通知
    要配置Jenkins以接收GitHub的WebHook通知,你需要在Jenkins上安装并配置GitHub插件,然后在GitHub仓库中设置WebHook指向Jenkins的URL。以下是配置步骤的简要说明和示例:确保Jenkins安装了GitHub插件。在Jenkins中配置GitHub插件,使其能够与GitHub通信(如......