Analyzing Billions of Rows of Log File Data
Analyzing Billions of Rows of Log File Data
2 Pagesnpm & OmniSci npm uses the OmniSci analytics database for exploring request log (log file) data. Request logs are a record of every request that the npm server has processed. In production, npm runs OmniSci on an Amazon EC2 r3.8xlarge instance and receives approximately 700 million events per day which are bulk-loaded hourly. Currently they keep a total of 10 days worth of data - approximately 7 billion rows. The log file data contains informa
