Commit Graph

28 Commits

Author SHA1 Message Date
Kim Lindberger
ebaa226853
elk7: 7.11.1 -> 7.16.1, 6.8.3 -> 6.8.21 + add filebeat module and tests (#150879)
* elk7: 7.11.1 -> 7.16.1

* nixosTests.elk: Improve reliability and compatibility with ELK 7.x

- Use comparisons in jq instead of grepping
- Match for `.hits.total.value` if version >= 7, otherwise it always
  passes
- Make curl fail if requests fails

* nixos/filebeat: Add initial module and test

Filebeat is an open source file harvester, mostly used to fetch logs
files and feed them into logstash.

This module can be used instead of journalbeat if used with
`filebeat7` and configured with the `journald` input.

* python3Packages.parsedmarc.tests: Fix breakage

- Don't use the deprecated elasticsearch7-oss package
- Improve jq query robustness and add tracing

* rl-2205: Note the addition of the filebeat service

* elk6: 6.8.3 -> 6.8.21

The latest version includes a fix for CVE-2021-44228.

* nixos/journalbeat: Add a loose dependency on elasticsearch

Avoid unnecssary back-off when elasticsearch is running on the same
host.
2021-12-17 00:20:52 +09:00
Robert Hensing
bbce1d9d60 nixosTests.elk.ELK-6: Fix evaluation 2021-11-03 15:15:47 -07:00
Robert Hensing
15e4c45cc7 nixos/tests/elk: Restructure attribute layout to avoid ELK-7 (non-OSS) on Hydra
... or any other tool that respects the recurseForDerivations attribute.
2021-11-03 12:24:41 +01:00
Robert Hensing
a9952f0093 Revert "nixos/tests/elk: remove OSS ELK-7 test"
This reverts commit 35db6bc554.
2021-11-03 11:41:53 +01:00
Andrew Childs
35db6bc554 nixos/tests/elk: remove OSS ELK-7 test 2021-11-03 11:37:07 +01:00
Andrew Childs
b7f4556002 nixos/tests/elk: propagate pkgs and system to test runner 2021-11-03 11:37:07 +01:00
Robert Hensing
843248d39f
Merge pull request #117379 from hercules-ci/nixos-metricbeat
nixos/metricbeat: init
2021-06-08 13:53:20 +02:00
talyz
3d9c3e5cfd
nixosTests.*: Don't use the -q flag with grep when used with curl
The `-q` flag makes grep close the pipe early, which curl doesn't
handle gracefully, but exits with an error like "(23) Failed writing
body".
2021-06-05 18:44:54 +02:00
Robert Hensing
5699d027ec nixos/metricbeat: init 2021-05-31 10:42:08 +02:00
Robert Hensing
9011d59758 logstash-*-oss: Add passthru.tests 2021-03-23 14:42:40 +01:00
Dominik Xaver Hörl
25bef2d8f9 treewide: simplify pkgs.stdenv.lib -> pkgs.lib
The library does not depend on stdenv, that `stdenv` exposes `lib` is
an artifact of the ancient origins of nixpkgs.
2021-01-10 20:12:06 +01:00
Bas van Dijk
dcb01353b3 nixos/tests/elk.nix: fix issue in the elasticsearch-curator
The elasticsearch-curator was not deleting indices because the indices
had ILM policies associated with them. This is now fixed by
configuring the elasticsearch-curator with `allow_ilm_indices: true`.

Also see: https://github.com/elastic/curator/issues/1490
2020-04-29 15:23:50 +02:00
Nikola Knezevic
900e288d22 Enable elasticsearch-curator on elasticsearch 7
The current nixpkgs use elasticsearch-curator 5.8.1. As of version 5.7.0,
elasticsearch-curator supports elasticsearch 7, thus this change enables tests
with ES 7.
2020-01-15 10:12:19 +01:00
Jacek Galowicz
8ecd07f9e3 nixosTests.elk: port to python 2020-01-08 10:51:57 +01:00
Jan Tojnar
f9237f3152
Merge branch 'master' into staging-next 2019-09-06 16:55:11 +02:00
adisbladis
0cce8856b2
elk-5: Remove tests
These packages are EOL and are about to be removed
2019-09-06 14:22:14 +01:00
volth
35d68ef143 treewide: remove redundant quotes 2019-08-26 21:40:19 +00:00
Bas van Dijk
477c552c7d nixos/journalbeat: support journalbeat >= 6 & add test 2019-05-10 15:41:41 +02:00
Bas van Dijk
13352f28d2 elk7: init at 7.0.0
This adds the following new packages:

+ elasticsearch7
+ elasticsearch7-oss
+ logstash7
+ logstash7-oss
+ kibana7
+ kibana7-oss
+ filebeat7
+ heartbeat7
+ metricbeat7
+ packetbeat7
+ journalbeat7

The default major version of the ELK stack stays at 6. We should
probably set it to 7 in a next commit.
2019-04-14 21:39:46 +02:00
Bas van Dijk
7062a073e8 elk: 6.5.1 -> 6.7.1 2019-04-09 12:34:01 +02:00
Florian Friesdorf
fbef5ab82f Remove myself as maintainer from packages
I'm currently not maintaining any packages.
2019-02-22 16:14:13 +01:00
Léo Gaspard
6c68fbd4e1
tests: refactor to carry the package set as an argument
This way, the package set will be possible to pass without re-importing
all the time
2018-11-11 23:11:45 +09:00
Bas van Dijk
241377ee76 nixos/tests/elk.nix: make sure the test doesn't wait for too long on elasticsearch-curator 2018-08-25 18:53:10 +02:00
David Smith
842000566b elasticsearch-curator: add test 2018-08-25 18:53:10 +02:00
Bas van Dijk
ebcdb822f8 elk: 6.2.4 -> 6.3.2
* The ELK stack is upgraded to 6.3.2.

* `elasticsearch6`, `logstash6` and `kibana6` now come with X-Pack which is
  a suite of additional features. These are however licensed under the unfree
  "Elastic License".

* Fortunately they also provide OSS versions which are now packaged
  under: `elasticsearch6-oss`, `logstash6-oss` and `kibana6-oss`.
  Note that the naming of the attributes is consistent with upstream.

* The test `nix-build nixos/tests/elk.nix -A ELK-6` will test the OSS
  version by default. You can also run the test on the unfree ELK using:
  `NIXPKGS_ALLOW_UNFREE=1 nix-build nixos/tests/elk.nix -A ELK-6 --arg enableUnfree true`
2018-07-28 00:01:31 +02:00
volth
2e979e8ceb [bot] nixos/*: remove unused arguments in lambdas 2018-07-20 20:56:59 +00:00
Bas van Dijk
803077ef1c elk: add elasticsearch6, logstash6, kibana6 and the beats at v6.1.0
This change is backwards compatible since the ELK tools at version 5.x
remain unchanged.

The test suite now both tests ELK-5 and ELK-6.
2018-01-02 01:15:29 +01:00
Bas van Dijk
2444eab485 ELK: update kibana and the elastic beats to 5.4 (#26252)
* Add kibana5 and logstash5
* Upgrade the elastic beats to 5.4
* Make sure all elastic products use the same version
  (see elk5Version)
* Add a test for the ELK stack
2017-06-13 22:36:08 +02:00