首页 > 其他分享 >CGCN Dataloader

CGCN Dataloader

时间:2023-06-20 18:56:22浏览次数:44  
标签:node graph Dataloader CGCN subjects dtype nodes data

from stellargraph.datasets import DatasetLoader


class ant_1_4(
    DatasetLoader,
    name="ant-1.4",
    directory_name="ant-1.4",
    url="",
    url_archive_format="",
    expected_files=[],
    description="",
    source="",
):
    _NUM_FEATURES = 20

    def load(
        self,
        directed=False,
        largest_connected_component_only=False,
        subject_as_feature=False,
        edge_weights=None,
        str_node_ids=False,
    ):
        nodes_dtype = str if str_node_ids else int

        return _load_defect_data(
            self,
            directed,
            largest_connected_component_only,
            subject_as_feature,
            edge_weights,
            nodes_dtype,
        )


def _load_defect_data(
    dataset,
    directed,
    largest_connected_component_only,
    subject_as_feature,
    edge_weights,
    nodes_dtype,
):
    # assert isinstance(dataset, (Ant))
    if nodes_dtype is None:
        nodes_dtype = dataset._NODES_DTYPE
    node_data = pd.read_csv("D:\\CGCN-main\\CGCN-main\\downstream_task\\data\\ant\\" + dataset.name + "\\Process-Binary.csv")
    edgelist = pd.read_csv(
        "D:\\CGCN-main\\CGCN-main\\downstream_task\\data\\ant\\" + dataset.name+ "\\edges.txt", sep="\t", header=None, names=["target", "source"], dtype=nodes_dtype
    )
    node_data.apply(pd.to_numeric, errors='ignore')

    # 0 to buggy, 1 to clean
    subjects_num = node_data['bug']
    label_list = subjects_num.to_list()
    labels = []
    for i in range(len(label_list)):
        if label_list[i] == 1:
            labels.append('buggy')
        else:
            labels.append('clean')
    subjects = pd.Series(labels, dtype='str')

    cls = StellarDiGraph if directed else StellarGraph
#   定义边的类型。
    features = node_data.iloc[:, 3:-1]
    # 第四列到倒数第二列的所有数据
    feature_names = node_data.iloc[:, 2]
    minMax = preprocessing.MinMaxScaler()
    features_std = minMax.fit_transform(features)

    graph = cls({"class": features_std}, {"to": edgelist})

    if edge_weights is not None:
        # A weighted graph means computing a second StellarGraph after using the unweighted one to
        # compute the weights.
        edgelist["weight"] = edge_weights(graph, subjects, edgelist)
        graph = cls({"class": node_data[feature_names]}, {"to": edgelist})

    if largest_connected_component_only:
        cc_ids = next(graph.connected_components())
        return graph.subgraph(cc_ids), subjects[cc_ids]

    return graph, subjects

标签:node,graph,Dataloader,CGCN,subjects,dtype,nodes,data
From: https://www.cnblogs.com/ZZXJJ/p/17494432.html

相关文章

  • CORA Dataloader 分析
    from.dataset_loaderimportDatasetLoaderclassCora(DatasetLoader,name="Cora",directory_name="cora",url="https://linqs-data.soe.ucsc.edu/public/lbc/cora.tgz",url_archive_format="gztar",......
  • Dataset & DataLoader
    fromtorch.utils.dataimportDataset,DataLoader1.DatasetThereare2differenttypesofdatasets:1.1map-styledatasets(mostcommonlyused)Representsamapfromindices/keystodatasamples.Forexample,suchadataset,whenaccessedwithdatase......
  • MegEngine 使用小技巧:借助 DataLoader 获取分批数据
    在使用MegEngine进行模型训练时,首先要进行的是数据加载和预处理。在此过程中,MegEngine中的megengine.data模块,提供了数据分批功能,其内部实现流程如下图:通过使用Datal......
  • TensorDataset和DataLoader
    一、TensorDataset语法:classtorch.utils.data.TensorDataset(data_tensor, target_tensor)作用:包装数据和目标张量(类似Python中的zip()函数),可通过第一维度索引两个张量恢......
  • pytorch不定长数据的dataloader读取
    参考资料:https://pytorch.org/docs/stable/data.html#dataloader-collate-fnhttps://blog.csdn.net/anshiquanshu/article/details/112868740在使用Pyt......
  • 12、dataloader的使用
    dataloader数据加载器:将数据加载到神经网络中。定义每次取多少数据,怎么取1、打开pytorch官网----Doc中找pytorch官方文档----搜索dataloader出现在torch.utils,.data.Dat......
  • Dataset和Dataloader的使用
    在深度学习中训练模型都是小批量小批量地优化训练的,即每次都会从原数据集中取出一小批量进行训练,完成一次权重更新后,再从原数据集中取下一个小批量数据,然后再训练再更新。......
  • Pytorch建模过程中的DataLoader与Dataset
      处理数据样本的代码会因为处理过程繁杂而变得混乱且难以维护,在理想情况下,我们希望数据预处理过程代码与我们的模型训练代码分离,以获得更好的可读性和模块......
  • PyTorch 深度学习实践第八讲(Dataset and DataLoader)
    上课代码importtorchimportnumpyasnpfromtorch.utils.dataimportDataset#Data是一个抽先类fromtorch.utils.dataimportDataLoaderclassDiabetesDataset(Datas......
  • 决策树(二):后剪枝,连续值处理,数据加载器:DataLoader和模型评估
    在上一篇文章中,我们实现了树的构造,在下面的内容中,我们将中心放在以下几个方面1.剪枝2.连续值处理3.数据加载器:DataLoader4.模型评估 一,后剪枝•为什么剪枝  –......