VictoriaLogs Practical Ingestion Guide for Message, Time and Streams

(victoriametrics.com)

59 points | by func25 3 days ago ago

17 comments

  • nklmilojevic 3 days ago ago

    Amazing product. We use VictoriaMetrics for quite a while, and previously used Loki and our custom Clickhouse/Vector approach for logs and we have switched to VictoriaLogs. It is much better and faster than Loki, same goes with custom CH/Vector thing we had. Kudos to the team, we are waiting for VictoriaTraces to switch Tempo instance to it for opentelemetry stuff.

    • CubsFan1060 3 days ago ago

      Can you talk a little bit about your Victoria Logs setup? About how many logs are you ingesting and what kind of sizing do you have on your setup?

      • nklmilojevic 3 days ago ago

        Sure thing!

        Ingested logs 24h: 428 Mil Ingested bytes 24h: 625GB Inser req/s: 6k/s

        8vCPU, 16GB mem. Running standard-rwo PVC on GCP.

        We have a couple of projects like this with similar usage and similar machine sizing.

        Still running vmlogs-single, and we will until we see a need to move to vmlogs-cluster version.

        • camel_gopher 3 days ago ago

          That sounds like a lot of resources provisioned for 6k/s. 625GB/24hr is a small footprint.

          • CubsFan1060 3 days ago ago

            I would have said it sounded pretty good. What technologies are you comparing it against, out of curiosity.

            • camel_gopher 3 days ago ago

              ClickHouse

              • CubsFan1060 3 days ago ago

                Can you share any additional details? What kind of ingestion do you have, with what dimensions of a clickhouse cluster?

                I'm also curious how it handles structured vs unstructured logs.

                Thanks!

        • CubsFan1060 3 days ago ago

          That seems pretty good. Do you have any sort of HA solution?

      • PaywallBuster 3 days ago ago

        personally Hetzner SX295 that has 14x 22 TB on a ZFS setup

        It ingests 70k lines per second without a sweat

        reads are just as fast

  • UncleFullstack 3 days ago ago

    Victoria logs is awesome and has put me at odds with my new devops hire who hates it and insists that Dynatrace can do the same thing for only $$$ a month. I tried using the their front end and it’s horrible. The API for Victoria is awesome and it’s fast.

  • paffdragon 3 days ago ago

    I use it in docker on a NAS - VictoriaMetrics, VictoriaLogs, Grafana - low resource usage, fast, so far zero issues.

  • pphysch 3 days ago ago

    We recently dropped-in VictoriaLogs for an old rsyslog setup. Extremely easy to deploy and admin, integrates nicely with Grafana. Definitely recommend doing this and moving to structured logging formats if you aren't already / are hesitant about the complexity of Elastic stack, etc.

    • chupasaurus 3 days ago ago

      There is no complexity in Elastic stack, only expensive footguns /s

  • ProofHouse 3 days ago ago

    I’m actually working on something right now where this can be extremely useful to me and I didn’t know about VictoriaLogs was using Loki. I wonder if anyone knows if there are other better alternatives or how this stacks up?