From 550e3884350717579775912276cab27ed25c7096 Mon Sep 17 00:00:00 2001 From: petya-vasileva Date: Sun, 5 Feb 2023 11:41:37 +0200 Subject: [PATCH 01/10] Adding pSDash project --- assets/logos/psdash.svg | 68 ++++++++++++++++++++++++++++++++++++++++ pages/projects/psdash.md | 46 +++++++++++++++++++++++++++ 2 files changed, 114 insertions(+) create mode 100644 assets/logos/psdash.svg create mode 100644 pages/projects/psdash.md diff --git a/assets/logos/psdash.svg b/assets/logos/psdash.svg new file mode 100644 index 000000000..f61ee6c88 --- /dev/null +++ b/assets/logos/psdash.svg @@ -0,0 +1,68 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/pages/projects/psdash.md b/pages/projects/psdash.md new file mode 100644 index 000000000..3f3f9512a --- /dev/null +++ b/pages/projects/psdash.md @@ -0,0 +1,46 @@ +--- +permalink: /projects/psdash.html +layout: project +title: pSDash +shortname: psdash +pagetype: project +image: logos/psdash.svg +blurb: Tracks network issues based upon the perfSONAR (and related) measurements +maturity: Deployed +maturity-note: +focus-area: osglhc +github: https://github.com/sand-ci/ps-dash +logowidth: 35% +start-date: 2022-10-01 +team: +- Marian Babik +- smckee +- petya-vasileva +- Ilija Vukotic +--- + + + +### Project description + + +The OSG and WLCG collaborations have been operating a global Research & Education (R&E) network monitoring infrastructure for more than 10 years. Currently data from over 230 perfSONAR instances are centrally gathered into an Elasticsearch cluster at the University of Chicago. This rich dataset should be able to provide visibility into our R&E networks and identify various types of network problems as they occur. + +[`pSDash`](https://ps-dash.uc.ssl-hep.org/) is designed to track, visualize, and filter for network issues based upon the perfSONAR and related network monitoring data being gathered by OSG and WLCG services. The project is complementary to the Alarms&Alerts service (for which anyone can subscribe; see below) and currently covers the following problems: +- **Throughput issues** +- **High packet loss** +- **Firewall issues** +- **Bad clock configurations** +- **Traceroute issues** +- **Divergence from the usual path** (based on the AS numbers) +- And more + +You can access the application at [https://ps-dash.uc.ssl-hep.org/](https://ps-dash.uc.ssl-hep.org/) and subscribe to receive emails about the most recent alarms at [https://aaas.atlas-ml.org/](https://aaas.atlas-ml.org/) (make sure you add a tag if you are interested in specific sites). + +[![GitHub Project](https://img.shields.io/badge/GitHub--blue?style=social&logo=GitHub)](https://github.com/sand-ci/ps-dash) pSDash code +[![GitHub Project](https://img.shields.io/badge/GitHub--blue?style=social&logo=GitHub)](https://github.com/sand-ci/AlarmsAndAlerts) Alarms&Alerts Service code + + +### Contact us + +Let us know if you find any issues or if you have comments/suggestions by opening a ticket with OSG support staff: email help@osg-htc.org and request support for networking/pSDash. From 27e5b1f9bbabd113092f0e046e1cf29b1148dc2b Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Sun, 5 Feb 2023 09:43:48 +0000 Subject: [PATCH 02/10] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- pages/projects/psdash.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/pages/projects/psdash.md b/pages/projects/psdash.md index 3f3f9512a..3beed64b8 100644 --- a/pages/projects/psdash.md +++ b/pages/projects/psdash.md @@ -24,7 +24,7 @@ team: ### Project description -The OSG and WLCG collaborations have been operating a global Research & Education (R&E) network monitoring infrastructure for more than 10 years. Currently data from over 230 perfSONAR instances are centrally gathered into an Elasticsearch cluster at the University of Chicago. This rich dataset should be able to provide visibility into our R&E networks and identify various types of network problems as they occur. +The OSG and WLCG collaborations have been operating a global Research & Education (R&E) network monitoring infrastructure for more than 10 years. Currently data from over 230 perfSONAR instances are centrally gathered into an Elasticsearch cluster at the University of Chicago. This rich dataset should be able to provide visibility into our R&E networks and identify various types of network problems as they occur. [`pSDash`](https://ps-dash.uc.ssl-hep.org/) is designed to track, visualize, and filter for network issues based upon the perfSONAR and related network monitoring data being gathered by OSG and WLCG services. The project is complementary to the Alarms&Alerts service (for which anyone can subscribe; see below) and currently covers the following problems: - **Throughput issues** @@ -37,8 +37,8 @@ The OSG and WLCG collaborations have been operating a global Research & Educatio You can access the application at [https://ps-dash.uc.ssl-hep.org/](https://ps-dash.uc.ssl-hep.org/) and subscribe to receive emails about the most recent alarms at [https://aaas.atlas-ml.org/](https://aaas.atlas-ml.org/) (make sure you add a tag if you are interested in specific sites). -[![GitHub Project](https://img.shields.io/badge/GitHub--blue?style=social&logo=GitHub)](https://github.com/sand-ci/ps-dash) pSDash code -[![GitHub Project](https://img.shields.io/badge/GitHub--blue?style=social&logo=GitHub)](https://github.com/sand-ci/AlarmsAndAlerts) Alarms&Alerts Service code +[![GitHub Project](https://img.shields.io/badge/GitHub--blue?style=social&logo=GitHub)](https://github.com/sand-ci/ps-dash) pSDash code +[![GitHub Project](https://img.shields.io/badge/GitHub--blue?style=social&logo=GitHub)](https://github.com/sand-ci/AlarmsAndAlerts) Alarms&Alerts Service code ### Contact us From 3ed9907adb3b2a9e6c36afd2c4cb2d7b3b876641 Mon Sep 17 00:00:00 2001 From: Petya Vasileva Date: Wed, 15 Mar 2023 12:28:15 +0200 Subject: [PATCH 03/10] Update petya-vasileva.yml --- _data/people/petya-vasileva.yml | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/_data/people/petya-vasileva.yml b/_data/people/petya-vasileva.yml index 1ccd3f01b..ed90d8057 100644 --- a/_data/people/petya-vasileva.yml +++ b/_data/people/petya-vasileva.yml @@ -8,3 +8,10 @@ shortname: petya-vasileva title: Ph.D Student (University of Plovdiv) website: presentations: + - title: Tools and methods for tracking network issues based on perfSONAR datasets + date: 2023-03-10 + url: https://drive.google.com/file/d/1UjNy80UlOKwVGx0GThM45TeCBTofd-G-/view + meeting: CI Engineering Lunch & Learn Series + focus-area: osglhc + project: psdash + location: Online From 378910dfdea2b0453fdec59fe80714c3376579f8 Mon Sep 17 00:00:00 2001 From: Petya Vasileva Date: Wed, 15 Mar 2023 12:31:09 +0200 Subject: [PATCH 04/10] Fix URL --- pages/projects/osg-networking.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pages/projects/osg-networking.md b/pages/projects/osg-networking.md index 2ac8065bd..341c9e389 100644 --- a/pages/projects/osg-networking.md +++ b/pages/projects/osg-networking.md @@ -39,7 +39,7 @@ The [OSG Network Monitoring](https://osg-htc.org/networking/) team designs, depl - **Toolkit Information Server:** We created a web service that serves as a central location for finding network related tools, documentation and applications at [ToolKitInfo](https://toolkitinfo.opensciencegrid.org/). -- **Network Data User Interfaces:** With the extensive set of network data we collect, we have also worked to provide various user interfaces to allow easy exploration and visualization of the data. We have created Kibana and Grafana dashboards and stand-alone applications like [TRACER](https://perfsonar.uc.ssl-hep.org/) and [pS-Dash](https://ps-dash.uc.ssl-hep.org/sites), all findable via the Toolkit Info link above. +- **Network Data User Interfaces:** With the extensive set of network data we collect, we have also worked to provide various user interfaces to allow easy exploration and visualization of the data. We have created Kibana and Grafana dashboards and stand-alone applications like [TRACER](https://perfsonar.uc.ssl-hep.org/) and [pSDash](https://ps-dash.uc.ssl-hep.org), all findable via the Toolkit Info link above. ### Collaborations From 4dfbbf6fd9471acfcdadf0959b8e562f15eedfe1 Mon Sep 17 00:00:00 2001 From: Petya Vasileva Date: Sat, 17 Jun 2023 08:58:00 +0200 Subject: [PATCH 05/10] Update petya-vasileva.yml --- _data/people/petya-vasileva.yml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/_data/people/petya-vasileva.yml b/_data/people/petya-vasileva.yml index ed90d8057..fb9cd0c85 100644 --- a/_data/people/petya-vasileva.yml +++ b/_data/people/petya-vasileva.yml @@ -15,3 +15,9 @@ presentations: focus-area: osglhc project: psdash location: Online + - title: Network problem detection and localisation using perfSonar measurements + date: 2023-06-30 + meeting: SoX Monthly Workshop + focus-area: osglhc + project: psdash, aaas + location: Online From a0eae5c50fa954634fc227ad8459263da7fb16e8 Mon Sep 17 00:00:00 2001 From: Petya Vasileva Date: Sun, 18 Jun 2023 23:43:29 +0300 Subject: [PATCH 06/10] Update petya-vasileva.yml --- _data/people/petya-vasileva.yml | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/_data/people/petya-vasileva.yml b/_data/people/petya-vasileva.yml index ed90d8057..9d93203fe 100644 --- a/_data/people/petya-vasileva.yml +++ b/_data/people/petya-vasileva.yml @@ -15,3 +15,10 @@ presentations: focus-area: osglhc project: psdash location: Online + - title: Network problem detection and localisation using perfSonar measurements + date: 2023-06-30 + meeting: SoX Monthly Workshop + focus-area: osglhc + project: psdash, aaas + location: Online + From a38e2811f9df90058fe22d31ae2ffee5da0ce5d8 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Sun, 18 Jun 2023 20:44:34 +0000 Subject: [PATCH 07/10] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- _data/people/petya-vasileva.yml | 1 - 1 file changed, 1 deletion(-) diff --git a/_data/people/petya-vasileva.yml b/_data/people/petya-vasileva.yml index 9d93203fe..fb9cd0c85 100644 --- a/_data/people/petya-vasileva.yml +++ b/_data/people/petya-vasileva.yml @@ -21,4 +21,3 @@ presentations: focus-area: osglhc project: psdash, aaas location: Online - From a4da164c6768e8ba897b206960310d7bc982af04 Mon Sep 17 00:00:00 2001 From: Petya Vasileva Date: Sun, 18 Jun 2023 23:53:09 +0300 Subject: [PATCH 08/10] Update petya-vasileva.yml --- _data/people/petya-vasileva.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/_data/people/petya-vasileva.yml b/_data/people/petya-vasileva.yml index fb9cd0c85..40fc6de07 100644 --- a/_data/people/petya-vasileva.yml +++ b/_data/people/petya-vasileva.yml @@ -17,6 +17,7 @@ presentations: location: Online - title: Network problem detection and localisation using perfSonar measurements date: 2023-06-30 + url: https://www.eventbrite.com/e/660376051547 meeting: SoX Monthly Workshop focus-area: osglhc project: psdash, aaas From c897d8e441da4854d9813453b896f6ea9b2f5e29 Mon Sep 17 00:00:00 2001 From: Petya Vasileva Date: Thu, 24 Oct 2024 13:59:02 +0200 Subject: [PATCH 09/10] Update petya-vasileva.yml Presentaions --- _data/people/petya-vasileva.yml | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/_data/people/petya-vasileva.yml b/_data/people/petya-vasileva.yml index ffa86cbb4..159ae4222 100644 --- a/_data/people/petya-vasileva.yml +++ b/_data/people/petya-vasileva.yml @@ -22,3 +22,27 @@ presentations: focus-area: osglhc project: psdash location: Online + - title: Network Problem Detection and Notification + date: 2023-06-14 + url: https://indico.cern.ch/event/1278716/contributions/5435153/attachments/2665686/4619355/NPDaN.pdf + meeting: ATLAS Software & Computing Week #75 + meetingurl: https://indico.cern.ch/event/1278716/#b-512111-adc-activities + focus-area: osglhc + project: osg-networking + location: CERN, Geneva, Switzerland + - title: perfSONAR Network Analytics - Status & Plans + date: 2024-10-11 + url: https://docs.google.com/presentation/d/1qrNFkyA9QmC5FSIQSGOfRrurzowcm08XQz3nnd9GtY8/edit?usp=sharing + meeting: #53 LHCOPN-LHCONE Meeting + meetingurl: https://indico.cern.ch/event/1410638/ + focus-area: osglhc + project: osg-networking + location: Beijing, China + - title: Enhancing Network Analytics through Machine Learning + date: 2024-10-23 + url: https://docs.google.com/presentation/d/1U0HN_U4RRd0kH1sgdh7gv_OoHKlaOguOiKzZZPbWhnI/edit?usp=sharing + meeting: CHEP 2024 + meetingurl: https://indico.cern.ch/event/1338689/ + focus-area: osglhc + project: osg-networking + location: Krakov, Poland From 9d322ffa92a39ad6656cac1c7430fecf754820a1 Mon Sep 17 00:00:00 2001 From: Petya Vasileva Date: Thu, 24 Oct 2024 14:34:35 +0200 Subject: [PATCH 10/10] Update petya-vasileva.yml --- _data/people/petya-vasileva.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/_data/people/petya-vasileva.yml b/_data/people/petya-vasileva.yml index 159ae4222..3bbfc18d7 100644 --- a/_data/people/petya-vasileva.yml +++ b/_data/people/petya-vasileva.yml @@ -25,7 +25,7 @@ presentations: - title: Network Problem Detection and Notification date: 2023-06-14 url: https://indico.cern.ch/event/1278716/contributions/5435153/attachments/2665686/4619355/NPDaN.pdf - meeting: ATLAS Software & Computing Week #75 + meeting: "ATLAS Software & Computing Week #75" meetingurl: https://indico.cern.ch/event/1278716/#b-512111-adc-activities focus-area: osglhc project: osg-networking @@ -33,7 +33,7 @@ presentations: - title: perfSONAR Network Analytics - Status & Plans date: 2024-10-11 url: https://docs.google.com/presentation/d/1qrNFkyA9QmC5FSIQSGOfRrurzowcm08XQz3nnd9GtY8/edit?usp=sharing - meeting: #53 LHCOPN-LHCONE Meeting + meeting: "#53 LHCOPN-LHCONE Meeting" meetingurl: https://indico.cern.ch/event/1410638/ focus-area: osglhc project: osg-networking @@ -41,7 +41,7 @@ presentations: - title: Enhancing Network Analytics through Machine Learning date: 2024-10-23 url: https://docs.google.com/presentation/d/1U0HN_U4RRd0kH1sgdh7gv_OoHKlaOguOiKzZZPbWhnI/edit?usp=sharing - meeting: CHEP 2024 + meeting: "CHEP 2024" meetingurl: https://indico.cern.ch/event/1338689/ focus-area: osglhc project: osg-networking