Hi,欢迎来到 卡瓦邦噶!我是 laixintao,现在生活在新加坡。我的工作是 SRE,喜欢在终端完成大部分工作,对各种技术都感兴趣。我从 2013 年开始写这个博客,写的内容很广泛,运维的方法论,编程的思考,工作的感悟,除了技术内容之外,还会分享一些读书感想,旅行游记,电影和音乐等。欢迎留下你的评论。

声明:本博客内容仅代表本人观点,和我的雇主无关。本博客承诺不含有 AI 生成的内容,所有内容未加说明均为博主原创,一经发布自动进入公有领域,本人放弃所有权利。转载无需本人同意。但是依然建议在转载的时候留下本博客的链接,因为这里的很多内容在发布之后会还会不断地继续更新和追加内容。 Not By AI

招聘启事二则

我就职的公司东海集团(Shopee 母公司)拓展 AI 云业务(neocloud),现在招聘计算、存储、网络方面的人才,是一个进入 AI Infra 领域很好的机会,以下是一个初级岗位和一个高级岗位的职位介绍。年限并不是硬性要求,有兴趣都可以发来简历试一试,邮箱是博客侧边栏的地址。这两个岗位目前都没有工作签证,要求应聘者必须在新加坡且有 PR 或 SC。

其一

Site Reliability Engineer, Compute and Storage (Graduate)

Level: Graduate / 0–2 years · Location: Singapore

About the team

Sea Cloud gives businesses and developers the infrastructure to build, train and run AI at scale: GPU compute, storage, networking, managed Kubernetes and model APIs, delivered as a public cloud.

The Compute and Storage SRE team delivers and provisions the platform’s foundation products and keeps them available, fast and durable: virtual machines, managed Kubernetes, block storage, object storage, and the traffic gateways that put them in front of customers. We combine software and systems engineering: we build the automation and observability that run these services, define and defend service-level objectives, write the standard operating procedures (SOPs) the platform runs on, and lead the response when things go wrong. We also prove resilience rather than assume it, through chaos engineering and platform-level disaster-recovery fire drills, and we harden the platform’s security and strengthen its compliance controls.

Join us to help build and run a cloud platform, where your work shapes how the service is run for years to come.

What you will do

  • Help operate the compute and storage products: virtual machines, managed Kubernetes, block storage, object storage and traffic gateways.
  • Deliver and provision infrastructure products: bring up new hosts, hypervisors, Kubernetes clusters and storage clusters, and take them from install through acceptance tests to service, using automated pipelines.
  • Implement improvements to reliability, efficiency and security, from small fixes to whole features, working from designs reviewed with senior engineers.
  • Build and tune dashboards and alerts for hypervisors, Kubernetes clusters, storage clusters and traffic gateways.
  • Write code (Python or Go) that automates operational work: health checks, capacity reports, node and volume lifecycle steps, runbook actions.
  • Turn each repeated manual fix into an SOP, then a script, then an automated remediation.
  • Write and maintain standard operating procedures (SOPs) for routine operations and known failures, and test that they work as written.
  • Take part in chaos experiments and disaster-recovery fire drills: prepare the run sheet, watch the signals, record what broke and what recovered.
  • Apply security hardening and compliance work: patch hosts and clusters, fix findings from vulnerability scans and hardening baselines, and collect the evidence audits need.
  • Respond to alerts, collect evidence, and contribute to incident reviews.

What you need

  • A degree in computer science, computer engineering or a related field, or equivalent practical experience.
  • Coding ability in at least one of Python, Go, C or Java, shown through coursework, internships or projects.
  • Working knowledge of Linux: processes, file systems, systemd, and the shell.
  • The basics of networking (TCP/IP, DNS, routing) and of how operating systems manage memory and I/O.
  • A habit of debugging from evidence: logs, metrics and reproduction, not guessing.
  • Clear written English; incident reviews and runbooks are part of the job.
  • Logical reasoning and critical thinking: you question assumptions, weigh the evidence, and can explain your conclusions.
  • Strong ownership: you see a problem through to the fix, not just to the handoff.
  • Eagerness to learn: you pick up new systems quickly and keep growing your skills.

Nice to have

  • An internship or project with containers, Kubernetes, Terraform or Ansible.
  • Exposure to virtualization (KVM, QEMU) or to a storage system such as Ceph, MinIO or ZFS.
  • Exposure to Prometheus, Grafana or another monitoring stack.
  • A homelab, open-source contribution or competitive programming record.

其二

Sea Cloud — Senior Site Reliability Engineer, Compute and Storage

Level: Senior / 6+ years
Location: Singapore

About the team

Sea Cloud gives businesses and developers the infrastructure to build, train and run AI at scale: GPU compute, storage, networking, managed Kubernetes and model APIs, delivered as a public cloud.

The Compute and Storage SRE team delivers and provisions the platform’s foundation products and keeps them available, fast and durable: virtual machines, managed Kubernetes, block storage, object storage, and the traffic gateways that put them in front of customers. We combine software and systems engineering: we build the automation and observability that run these services, define and defend service-level objectives, write the standard operating procedures (SOPs) the platform runs on, and lead the response when things go wrong. We also prove resilience rather than assume it, through chaos engineering and platform-level disaster-recovery fire drills, and we harden the platform’s security and strengthen its compliance controls.

Join us to help build and run a cloud platform, where your work shapes how the service is run for years to come.

What you will do

  • Own the reliability of one or more compute and storage products, such as virtual machines, managed Kubernetes, block storage, object storage or traffic gateways. You set their SLOs, agree error budgets with the product teams, and report against them.
  • Own the delivery and provisioning of infrastructure products: design repeatable, automated pipelines that bring up new capacity, regions and availability zones (hosts, hypervisors, Kubernetes and storage clusters, traffic gateways), with acceptance tests before anything goes into service.
  • Design and implement systems that continually improve the reliability, efficiency and security of the compute and storage platform, from architecture through to production code.
  • Review new designs and launches for failure modes, data durability, observability and recovery before they reach customers.
  • Design and build the reliability tooling the team runs on: automated detection and remediation, safe rollouts with canaries and automatic rollback, and capacity forecasting for compute and storage.
  • Own the SOP library for your products: make every SOP current, tested in drills, and automated where it can be.
  • Lead the chaos engineering programme: choose the failure modes that matter (host, rack, disk, network partition, gateway, control plane), design experiments with a bounded blast radius, and turn findings into fixes.
  • Plan and run platform-level disaster-recovery fire drills, up to the loss of an availability zone. Measure recovery against RTO and RPO targets and keep the evidence auditors will ask for.
  • Drive security hardening and compliance for your products: define hardening baselines for hosts, hypervisors, Kubernetes and storage (for example CIS benchmarks), run vulnerability and patch management, enforce least-privilege access and encryption, and turn compliance controls into automated checks and audit evidence.
  • Lead major incidents as incident commander, run blameless post-incident reviews, and drive the fixes to closure.
  • Mentor graduate engineers and shape the on-call rotation.

What you need

  • 6+ years in SRE, production engineering, infrastructure software engineering, or a related field, on large distributed systems.
  • Strong coding in Go or Python. You have shipped production tools and services, not only scripts.
  • Production experience operating at least one of: a virtualization platform, Kubernetes at scale, or a distributed storage system.
  • Deep Linux skills: kernel, file system and block I/O behaviour, performance analysis, and debugging under pressure.
  • Infrastructure as code (Terraform or similar) and building an observability stack, not only using one.
  • A track record of leading incidents and of running failure tests or DR exercises in production.
  • Working knowledge of datacenter networking: L3 fabrics, BGP, load balancing, DNS.
  • Logical reasoning and critical thinking: you question assumptions, weigh the evidence, and can explain your conclusions.
  • Strong ownership: you see a problem through to the fix, not just to the handoff.
  • Eagerness to learn: you pick up new systems quickly and keep growing your skills.

Nice to have

  • Virtualization internals: KVM, QEMU, libvirt, live migration, or a cloud stack such as OpenStack.
  • Distributed storage internals: Ceph (RBD, RGW), an S3-compatible object store, erasure coding, replication and rebuild behaviour.
  • Gateways and load balancers in front of cloud APIs: S3 gateways, API gateways, L4/L7 load balancers, rate limiting.
  • Chaos engineering tools such as Chaos Mesh, LitmusChaos or Gremlin, or a fault-injection framework you built.
  • Security hardening in practice: CIS benchmarks, OS and Kubernetes hardening, secrets management, policy-as-code (OPA, Kyverno).
  • Running a public cloud service with external customers, SLAs and compliance audits (ISO 27001, SOC 2, MTCS).
  • GPU servers or AI workloads on the compute and storage you run.

 

消费理念

在一次公司的培训上,一个同事带了一个塑料水瓶,我看见过这种塑料瓶子很多次,我问他是从哪里买的。他说,新加坡人在上学的时候基本每人都有一个,这家公司 make product for generations,可以用很久。他的这个水瓶是他上学的时候买的。

Make product for generations. 这也是我喜欢的理念。

去年买过一个电纸书(什么牌子的并不重要,主要是我想读微信读书里面的资源,随便找了一个支持微信读书的),然后这一年来完成了微信的阅读挑战,即读了 300 个小时,持续 360 天。感觉非常值。

电纸书

在 10 年前买过一个 HHKB 键盘,白色的外壳已经发黄了,但是所有的功能仍完好,今天依然在服役。HHKB 也是一个好产品,键位设计的合理,基本没有按键是闲置浪费的,也没有按键是缺失的。

我只用过 3 个 iphone,iphone 6s,iphone 12 mini,以及现在在用的 iphone 16 pro。iphone 也是很好的产品,事实上的很耐用。不过我现在在手机上的时间少了,我基本不看社交媒体了,微信朋友圈也不看,用的最多的软件是 chatGPT 和 youtube 看视频。身边的人比网上的人要重要的一些。应该多花点时间在线下的社交。手机的操作效率比电脑低多了,屏幕小,也没有键盘,所以我用电脑比较多。

在买东西上,我经常觉得应该这么算账:假设一部手机售价是 5000 元,这只是一个账面够买价值。实际你支出的价值是 (5000 元 – 最后的剩余成本) / 使用时间。比如,5000 元购买,使用1年之后,以 3000 元售出,那么实际成本是 2000 元/年。但是如果和我一样,5000元购买之后,用了4年,用完之后放在那里也懒得卖掉了,成本就是 5000 元/4年 = 1250元/年。考虑到使用的时间,就可以很轻松地得到结论:我们应该坚持长期主义,购买可以用很久的商品。

这样一来,购买时候的成本多几百少几百就不怎么重要了。要看自己之后是不是会长期使用才比较重要。

第二个消费理念是:商品是服务于人的。所以我不喜欢购买超出自己需要的、买有经济压力的昂贵商品。

已经发黄的 HHKB 键盘,拍照的时候感觉需要清洗一下了。对比刚买的时候,见这里1

比如说,我用手机一般不会贴膜和戴手机壳,拿起来很不舒服,但是前提是不会买超出负担能力的贵手机。这样磕了也不会心疼。

比如说背包,我也不会买很贵的,我会买实用的,又可以用比较久的。

这样决策成本也比较低,即使买来不合适,投入的成本也比较低。可以后悔。

很多商品的边际效用随价格上涨递减非常快,意思是,达到 80% 的品质只需要一个稍贵的价格即可,但是达到 90% 就可能需要额外付出 50% 的成本。原因一般是技术成本较高,以及市场不大,就需要很少的目标用户去平摊昂贵的开发成本。我一般不去平摊这些开发成本,因为我知道自己不需要。

想到这里是因为今天看了一个视频:Signs upgrading will make your life worse2,买更好的东西会让你失去自由。比如买了一辆很贵的车,有一点划痕心里就会很难受。

除了买比较贵的东西,买不需要的东西也会增加负担。买了本不需要的东西,由于前面那条理念的存在,需要持续使用才能降低持有成本,这样也会失去一部分自由。

说到自由,我也基本不考虑参加积分类型的会员机制,对于促销,基本只看考虑当下消费立减可以便宜多少钱。积分类型的会员促销也会失去一部分消费自由。

什么时候需要真正地付出更高的价格去升级呢?一般来说我能清楚地知道下一级别的商品会给自己带来什么,比如技术的提供,比如现在遇到了瓶颈。举一个反例,我用菜刀就从没遇到过这种瓶颈,强如王刚也说,菜刀是磨出来的,买什么样的菜刀都是一样,主要靠磨。我发现确实是这样,花 10 块钱买一个磨石比花 200 买一把好刀更重要。

这样的消费理念伴随我多年,works for me.

(不过最近几年的物欲极低,应该是我的兴趣都是在软件上面了,硬件上面只要一台电脑就够了)

  1. HHKB开箱 ↩︎
  2. https://www.youtube.com/watch?v=dK7YgDYlIok ↩︎

 

游戏

一起去上班路上:

欣:你喜欢玩什么游戏?

我:啥?

欣:我同事聊起来游戏,他们问我你玩什么游戏。

我:那一定是魂系游戏,艾尔登法环,血缘诅咒,黑暗之魂。

欣:那是啥?我说你平时玩的多的是那个抓鱼的,潜水员戴夫,超级玛丽,马里奥赛车。

……