Third-Party Connectivity for SonarG

SonarG data is stored within the SonarW NoSQL Data Warehouse. SonarW is wire-level compatible with MongoDB and as such SonarG data can be accessed by any tool or application that supports MongoDB.

Additionally, SonarG includes the SonarSQL layer - a software layer through which you can access SonarG data using SQL SELECT statements. SonarSQL supports SELECT statements and is compatible with the MySQL dialect. As such, you can access SonarG data from tools and applications that can connect to MySQL.

Of the two, NoSQL access through MongoDB drivers is preferred.

Home Grown Application Access

Access the SonarG data from applications using any programming language for which a MongoDB driver exists including C, C++, C#, Java, Python, Ruby, Scala, Perl and more. For a complete list see http://docs.mongodb.org/ecosystem/drivers/

You can also access SonarG data from applications using a MySQL driver. As mentioned, using a MongoDB driver is recommended and easier to use.

Tool Access

Access the SonarG data using any third party tool that can access MongoDB or that can access MySQL. This includes all mainstream BI tools. For a partial list of tools for MongoDB see http://mongodb-tools.com/.

Access from R

To access SonarG data from the R environment use the jSonarR package which you can download from CRAN directly in R. The package is available at https://cran.r-project.org/web/packages/jSonarR/index.html.

Data can be accessed directly as JSON or as an R data frame on which any existing R algorithm and package can operate. A simple program can look like this:

install.packages("jSonarR")
library('jSonarR')

con <- jSonarR::new.SonarConnection('https://localhost:8443', '192.168.1.123',
'gdm3', sdb="lmrm__sonarg", port=27117, username="", pwd="")

failed_sqls = sonarFind(con, 'sonarg_q', 'failed_by_ips')

plot(failed_sqls)

Line 1 downloads and installs the package and line 2 loads the library.

Line 3 creates a connection object using the Web Services layer (part of SonarG).

Line 4 makes the call, retrieves the data from SonarG, and creates an R data frame.

Line 5 is an example of a use of an R function on the dataframe. Any package can be used since failed_sqls is just a standard R data frame.

Access from Hadoop

Use the MongoDB connector for Hadoop to access data stored in SonarG from Hadoop services. See https://docs.mongodb.com/ecosystem/tools/hadoop/.

Analytic Libraries, Machine Learning and More

Any Machine Learning and analytics library such as NLTK, Scikit-learn, PyML, libDAI, Armadillo and many more may be used - simply use one of the existing MongoDB drivers to access the data. Additionally, numerous open-source projects exist to make this access even easier, two leading examples being:

Producing Events/Data to be sent to Syslog

SonarG can stream any data set as syslog messages. Select any pipeline that you want to generate syslog messages and ensure that you project a field called syslog. This field should format the data as you wish to push it to syslog. Then schedule/dispatch it as a job with “syslog” in the copy-to section. Set syslog rules on the SonarG instance to determine where to forward these syslog message.

Producing Events/Data to be sent to Kafka

SonarG can produce data and publish it to Kafka topics. Select any pipeline that you want to generate Kafka messages from, and ensure that you project fields called key and value. The value field should format the data as you wish to push it to the topic. Then schedule/dispatch it as a job selecting Kafka as the type and defining what Kafka nodes and topics to publish it to as follows:

  1. Enter a name in the ‘Name’ field.
  2. Enter a valid cron string in the ‘Cron string’ field.
  3. Select ‘Kafka’ as the ‘Output type’.
  4. In the ‘Kafka Servers’ field enter an ip:port (for example, 172.98.2.166:9090) for Kafka servers with no authentication. Alternatively, in order to authenticate using Kerberos or SSL, an alias must be defined with the relevant parameters in /etc/sonar/sonar_kafka_aliases.yaml (the file includes examples of all optional aliases) on the SonarG server. Once such an alias is defined, enter the alias name enclosed in brackets in the ‘Kafka Servers’ field (for example, [test-alias]). You can also create aliases for non-authenticated kafka servers.
  5. Enter a list of topics separated by commas in the ‘Destinations to copy to’ field.

Example form:

_images/Kafka_example.png