Пожалуйста, используйте этот идентификатор, чтобы цитировать или ссылаться на этот ресурс:
http://earchive.tpu.ru/handle/11683/52921
Полная запись метаданных
Поле DC | Значение | Язык |
---|---|---|
dc.contributor.author | Alekseev, Aleksandr Aleksandrovich | en |
dc.contributor.author | Barreiro Megino, F. G. | en |
dc.contributor.author | Klimentov, A. A. | en |
dc.contributor.author | Korchuganova, T. A. | en |
dc.contributor.author | Maendo, T. | en |
dc.contributor.author | Padolski, S. V. | en |
dc.date.accessioned | 2019-02-28T08:08:18Z | - |
dc.date.available | 2019-02-28T08:08:18Z | - |
dc.date.issued | 2018 | - |
dc.identifier.citation | Building analytical platform with Big Data solutions for log files of PanDA infrastructure / A. A. Alekseev [et al.] // Journal of Physics: Conference Series. — Bristol : IOP Publishing, 2018. — Vol. 1015 : Information Technologies in Business and Industry (ITBI2018) : International Conference, January 17-20, 2018, Tomsk, Russian Federation : [proceedings]. — [032003, 6 p.]. | en |
dc.identifier.uri | http://earchive.tpu.ru/handle/11683/52921 | - |
dc.description.abstract | The paper describes the implementation of a high-performance system for the processing and analysis of log files for the PanDA infrastructure of the ATLAS experiment at the Large Hadron Collider (LHC), responsible for the workload management of order of 2M daily jobs across the Worldwide LHC Computing Grid. The solution is based on the ELK technology stack, which includes several components: Filebeat, Logstash, ElasticSearch (ES), and Kibana. Filebeat is used to collect data from logs. Logstash processes data and export to Elasticsearch. ES are responsible for centralized data storage. Accumulated data in ES can be viewed using a special software Kibana. These components were integrated with the PanDA infrastructure and replaced previous log processing systems for increased scalability and usability. The authors will describe all the components and their configuration tuning for the current tasks, the scale of the actual system and give several real-life examples of how this centralized log processing and storage service is used to showcase the advantages for daily operations. | en |
dc.language.iso | en | en |
dc.publisher | IOP Publishing | en |
dc.relation.ispartof | Journal of Physics: Conference Series. Vol. 1015 : Information Technologies in Business and Industry (ITBI2018). — Bristol, 2018. | en |
dc.rights | info:eu-repo/semantics/openAccess | en |
dc.subject | платформы | ru |
dc.subject | Big Data | ru |
dc.subject | лог-файлы | ru |
dc.subject | инфраструктура | ru |
dc.subject | высокопроизводительные системы | ru |
dc.subject | программное обеспечение | ru |
dc.subject | данные | ru |
dc.subject | обработка | ru |
dc.subject | хранение | ru |
dc.title | Building analytical platform with Big Data solutions for log files of PanDA infrastructure | en |
dc.type | Conference Paper | en |
dc.type | info:eu-repo/semantics/publishedVersion | en |
dc.type | info:eu-repo/semantics/conferencePaper | en |
dcterms.audience | Researches | en |
local.description.firstpage | 32003 | - |
local.filepath | http://dx.doi.org/10.1088/1742-6596/1015/3/032003 | - |
local.identifier.bibrec | RU\TPU\network\28050 | - |
local.identifier.perskey | RU\TPU\pers\37959 | - |
local.localtype | Доклад | ru |
local.volume | 10152018 | - |
local.conference.name | Information Technologies in Business and Industry (ITBI2018) | - |
local.conference.date | 2018 | - |
dc.identifier.doi | 10.1088/1742-6596/1015/3/032003 | - |
Располагается в коллекциях: | Материалы конференций |
Файлы этого ресурса:
Файл | Описание | Размер | Формат | |
---|---|---|---|---|
dx.doi.org-10.1088-1742-6596-1015-3-032003.pdf | 499,59 kB | Adobe PDF | Просмотреть/Открыть |
Все ресурсы в архиве электронных ресурсов защищены авторским правом, все права сохранены.