首页 > 系统相关 >C#内存缓存链表BytesListBuffer

C#内存缓存链表BytesListBuffer

时间:2023-10-15 21:11:56浏览次数:41  
标签:count C# 链表 int copySize offset Position BytesListBuffer public

C#自带MemoryStream,可以作为内存缓存使用,用来存储byte[]数据,但是MemoryStream的扩展机制是通过获取整块连续内存来缓存数据,当需要缓存较大数据时,虽然空闲内存可能足够,但是可能找不到足够大的整块连续内存而导致扩展失败产生out of memory的异常。另外,对于很多缓存场景,重新分配整块内容,并将原缓存内容拷贝到新缓存,也会产生性能问题,而且使用链表缓存方式,通过将多块内存结合作为一个大缓存使用,能提升效率,也能解决大块内存不足问题。

BytesListBuffer将缓存作为内存块的链表保存,可以指定块大小,当所需缓存超过块大小时,会申请更多的块来保存数据,而原数据块保持不变。

BytesListBuffer继承自Stream,操作方法跟MemoryStream基本一致,读数据:

public override int Read(byte[] buffer, int offset, int count)
        {
            count = (int)Math.Min(_length - Position, count);

            int totalRead = 0;
            int copySize = 0;
            while (count > 0)
            {
                copySize = (int)Math.Min(count, BlockSize - blockPos);
                Buffer.BlockCopy(block, (int)blockPos, buffer, offset, copySize);

                Position += copySize;
                count -= copySize;
                offset += copySize;
                totalRead += copySize;
            }
            return totalRead;
        }

写数据:

public override void Write(byte[] buffer, int offset, int count)
        {
            if (BeforeWrite?.Invoke(count) == false)
                return;

            long beginPos = Position;
            int copySize;
            try
            {
                while (count > 0)
                {
                    copySize = Math.Min(count, (int)(BlockSize - blockPos));
                    Buffer.BlockCopy(buffer, offset, block, (int)blockPos, copySize);

                    _length = Math.Max(_length, Position + copySize);
                    Position += copySize;

                    count -= copySize;
                    offset += copySize;
                }
            }
            catch
            {
                Position = beginPos;
                throw;
            }
        }

默认块大小:

public long BlockSize = 64 * 1024;

默认块大小可以在对象初始化时根据需要调整,64K在大多数情况下有很好的效率。

完整代码:BytesListBuffer.cs

using System;
using System.Collections.Generic;
using System.IO;

namespace util
{
    public class BytesListBuffer : Stream
    {
        public long BlockSize = 64 * 1024;
        public List<byte[]> Blocks = new List<byte[]>();

        public Func<int, bool> BeforeWrite;

        public override bool CanRead => true;
        public override bool CanSeek => true;
        public override bool CanWrite => true;

        long _length = 0;
        public override long Length => _length;

        long _position = 0;
        public override long Position
        {
            get => _position;
            set => _position = value;
        }

        protected byte[] block
        {
            get
            {
                while (Blocks.Count <= blockIdx)
                {
                    Blocks.Add(new byte[BlockSize]);
                }
                return Blocks[(int)blockIdx];
            }
        }
        protected long blockIdx => Position / BlockSize;
        protected long blockPos => Position % BlockSize;

        public override void Flush() { }

        public override int Read(byte[] buffer, int offset, int count)
        {
            count = (int)Math.Min(_length - Position, count);

            int totalRead = 0;
            int copySize = 0;
            while (count > 0)
            {
                copySize = (int)Math.Min(count, BlockSize - blockPos);
                Buffer.BlockCopy(block, (int)blockPos, buffer, offset, copySize);

                Position += copySize;
                count -= copySize;
                offset += copySize;
                totalRead += copySize;
            }
            return totalRead;
        }

        public override void Write(byte[] buffer, int offset, int count)
        {
            if (BeforeWrite?.Invoke(count) == false)
                return;

            long beginPos = Position;
            int copySize;
            try
            {
                while (count > 0)
                {
                    copySize = Math.Min(count, (int)(BlockSize - blockPos));
                    Buffer.BlockCopy(buffer, offset, block, (int)blockPos, copySize);

                    _length = Math.Max(_length, Position + copySize);
                    Position += copySize;

                    count -= copySize;
                    offset += copySize;
                }
            }
            catch
            {
                Position = beginPos;
                throw;
            }
        }

        public override long Seek(long offset, SeekOrigin origin)
        {
            switch (origin)
            {
                case SeekOrigin.Begin:
                    Position = offset;
                    break;
                case SeekOrigin.Current:
                    Position += offset;
                    break;
                case SeekOrigin.End:
                    Position = Length - offset;
                    break;
            }
            return Position;
        }

        public override void SetLength(long value)
        {
            _length = value;
        }

        public byte[] ToArray()
        {
            long pos = Position;
            Position = 0;
            byte[] dst = new byte[Length];
            Read(dst, 0, (int)Length);
            Position = pos;
            return dst;
        }

        public void ReadFrom(Stream rd, long count)
        {
            byte[] buff = new byte[4096];
            int read;
            do
            {
                read = rd.Read(buff, 0, (int)Math.Min(4096, count));
                count -= read;
                this.Write(buff, 0, read);
            }
            while (count > 0);
        }

        public void WriteTo(Stream wrt)
        {
            long pos = Position;
            Position = 0;
            this.CopyTo(wrt);
            Position = pos;
        }

        //void check(byte[] buffer, int offset, int count)
        //{
        //    if (count < 0)
        //        throw new ArgumentOutOfRangeException();
        //    if (buffer == null)
        //        throw new ArgumentNullException();
        //    if (offset < 0)
        //        throw new ArgumentOutOfRangeException();
        //}
    }
}
View Code

Github链接:

https://github.com/bsmith-zhao/vfs/blob/main/util/BytesListBuffer.cs

标签:count,C#,链表,int,copySize,offset,Position,BytesListBuffer,public
From: https://www.cnblogs.com/bsmith/p/17766191.html

相关文章

  • Mybatis使用SELECT LAST_INSERT_ID()返回0问题避坑
    Mybatis使用SELECTLAST_INSERT_ID()返回0问题避坑SELECTLAST_INSERT_ID()用于返回最后插入表中数据的主键值,一般用于表主键自增且需要用到该自增的主键值的情况<insertid="insertOrder"parameterType="com.example.bobosapce.Entity.WorkOrder">INSERTINTOWORKOR......
  • 常用cmd命令
    calc:启动计算器 Shutdown-s-t30:表示30秒后自动关机,中间带有空格。shutdown-a:取消定时关机Shutdown-r-t30:表示30秒后自动重新启动control:控制面版devmgmt.msc:设备管理器desk.cpl:屏幕辨别率diskmgmt.msc:磁盘管理dxdiag:检查DirectX信息explorer:资源管理器fsmgmt.......
  • moectf wp
    REReverse入门指北if(*(_DWORD*)v7==13)sub_401082(aMoectfF1rstSt3,v6);aMoectfF1rstSt3db'moectf{F1rst_St3p_1s_D0ne}',0Ah,0base_64pycdc下载编译发现是base64变种http://web.chacuo.net/netbasex把索引表复制进去解密拿到flagUPX!exeinfope中看......
  • Docker note
    1.1Docker服务相关命令启动dockers服务:systemctlstartdocker停止dockers服务:systemctlstopdocker重启dockers服务:systemctlrestartdocker查看dockers服务状态:systemctlstatusdocker设置开机启动docker服务:systemctlenabledocker1.2镜像相关命令1.2.1查......
  • Scala语言入门:初学者的基础语法指南
    本文已收录至GitHub,推荐阅读......
  • shctf week1 wp
    REez_asm程序的逻辑大概是把输入的数据flag按字节^0x1E-0x0A一遍,然后输出,所以只需要置反一下先+0x0A然后再^0x1e就能求出flag.text:0000000000401566loc_401566:;CODEXREF:main+65↓j.text:0000000......
  • java.lang.TypeNotPresentException: Type javax.servlet.http.HttpServletRequest no
    完整的报错信息java.lang.TypeNotPresentException:Typejavax.servlet.http.HttpServletRequestnotpresent atjava.base/sun.reflect.generics.factory.CoreReflectionFactory.makeNamedType(CoreReflectionFactory.java:117)~[na:na] atjava.base/sun.reflect.generics......
  • access MD5加密
    PrivateConstBITS_TO_A_BYTE=8PrivateConstBYTES_TO_A_WORD=4PrivateConstBITS_TO_A_WORD=32Privatem_lOnBits(30)Privatem_l2Power(30)PrivateFunctionLShift(lValue,iShiftBits)IfiShiftBits=0ThenLShift=lValueExitFunction......
  • 文心一言 VS 讯飞星火 VS chatgpt (113)-- 算法导论10.2 5题
    五、用go语言,使用单向循环链表实现字典操作INSERT、DELETE和SEARCH,并给出所写过程的运行时间。文心一言:在Go语言中,我们首先需要定义一个Node结构体来表示单向循环链表的节点,然后再定义一个LinkedList结构体来表示单向循环链表。接下来,我们可以实现INSERT、DELETE和SEARCH操作。......
  • Easysearch压缩模式深度比较:ZSTD+source_reuse的优势分析
    引言在使用Easysearch时,如何在存储和查询性能之间找到平衡是一个常见的挑战。Easysearch具备多种压缩模式,各有千秋。本文将重点探讨一种特别的压缩模式:zstd+source_reuse,我们最近重新优化了source_reuse,使得它在吞吐量和存储效率方面都表现出色。测试概览测试条件选用了esr......