esProc SPL é um motor de análise e computação de código aberto baseado em JVM para dados estruturados e semi-estruturados. Especificamente, SPL permite que você escreva de forma simples e torna a execução mais rápida; SPL pode ser usado de forma independente ou embutido em aplicações. O esProc SPL é aplicado principalmente em cenários como substituição de data warehouse para realizar trabalhos em lote off-line e consultas on-line, servindo como middleware para micro-serviços orientados a dados, substituindo procedimentos armazenados, simplificando cálculos de relatórios, computação mista em tempo real, computação em arquivos.
O esProc SPL fornece um ambiente de desenvolvimento simples e fácil de usar com funções de depuração completas, permitindo que você codifique passo a passo e visualize o resultado de execução de cada etapa em tempo real. SPL é um sistema de sintaxe especialmente projetado e suporta naturalmente cálculos passo a passo, e cálculos procedurais complexos em particular. SPL possui uma biblioteca de computação de dados rica embutida, incluindo cálculos de string, data e hora, cálculos matemáticos, leitura/escrita de arquivos e banco de dados, análise de dados em camadas JSON/XML, e modelagem e previsão de IA.
O esProc SPL tem uma integração muito alta, podendo funcionar de forma independente ou ser integrado de forma transparente em aplicações para servir como um motor de computação dentro da aplicação, desempenhando um papel importante em cenários como micro-serviços, computação de borda e preparação de dados para relatórios. O esProc SPL suporta diversas fontes de dados, incluindo dezenas de fontes de dados como MongoDB, Elasticsearch, Hbase, HDFS e Influxdb. Esses dados podem ser calculados diretamente ou de forma mista sem carregá-los no banco de dados. Além disso, SPL fornece seu próprio armazenamento de arquivos de dados eficiente, esses formatos de dados privados não apenas aumentam o desempenho, mas permitem que você armazene dados com base na categoria de negócios em diretórios de árvore do sistema de arquivos.
Além de trabalhos em lote off-line e consultas on-line, o esProc SPL também suporta mais cenários de aplicação: micro-serviços orientados a dados, substituição de procedimentos armazenados, eliminação de tabelas intermediárias de bancos de dados, atendimento a requisitos intermináveis de desenvolvimento de relatórios, roteamento de dados programável para implementar cálculos de front-end, computação mista para implementar HTAP em tempo real, e realização de computação em arquivos para implementar Lakehouse.
0.0
Nps Score
Já tem esProc SPL Community?
Todas as Discussões de esProc SPL Community
# Macros in SPL
refers http://c.raqsoft.com/article/1687916213139
In addition to common static code, sometimes dynamic code is also needed to solve problems, such as generating code (or part of code) based on parameters and dynamically executing it. For programming languages that lack dynamic coding mechanisms, it is usually necessary to write the variable parts of the code in string form. For example, when referencing dataset field names in Python, it is necessary to write them as strings to achieve the effect of dynamic code. However, this will make it inconvenient to read and write more common static code. SQL, on the other hand, can directly write field names (as well as filter conditions, grouping expressions, etc.) in the code without having to write them into strings, making it easier to read and write static code, but it is difficult to handle dynamic code.
SPL inherits the SQL style of static code, allowing for direct writing of code parts, such as field names, without the need to be written as strings. In addition, SPL also provides macros to achieve dynamic code effects.
Example 1: Dynamically sort the order table based on the parameter pSortList, which contains an indefinite number of sorting fields separated by commas.
This dynamic code can be implemented using SPL macros: T("Orders.txt").sort(${pSortList})
##### Post Metadata
- Posted at: há mais de 2 anos
- Author title: Marketing Specialist at Scudata
- Net upvotes: 1
# the challenger of SQLite
Many small and micro applications also require certain data processing and computing abilities. Integrating a database will make the application too heavy. In this case, SQLite is a good choice for its simple framework, easy-integration, ability of storing data persistently and, it provides SQL to implement calculation.
However, for some more complex scenarios, SQLite still falls short.The shortcomings of SQLite in the face of complex scenarios:
->Data source support
->Complex calculation
->Flow processing
Use esProc SPL to solve the difficulties of SQLite:
->Data source support
SPL only takes one step to read csv files in SPL, and has built-in multiple data source interfaces, including tsv, xls, Json, XML, RESTful, WebService and other databases, and even supports special data sources such as Elasticsearch and MongoDB. SPL can directly parse multi-layer data sources. SPL has good openness and can directly calculate multiple data sources; we can perform cross-source computing between these data sources and SPL bin file. In addition to supporting the persistence of data to its own formats, SPL also supports the persistence of data to other data sources. Likewise, SPL uses table sequence as medium.
->Computing ability
SPL supports ordered computing, set computing, stepwise computing and association computing. By means of these abilities, complex structured data computation can be simplified.
->Flow processing
SPL comes with flow control statements. Using such statements together with the table sequence objects built in SPL can conveniently achieve complete business logic.
details refer to http://c.raqsoft.com/article/1686896755818
##### Post Metadata
- Posted at: há mais de 2 anos
- Net upvotes: 1
# Multiple desktop tools that allow querying data files directly in SQL
In our daily work, we often need to query and analyze data files (such as csv/xls file), most of which have quite regular form (each row corresponds to one record) and can be regarded as a database table, and it will be relatively convenient to calculate such files in SQL. However, we often have to import these files into database before using SQL, which is troublesome and difficult, and not friendly to non-professional programmers. Given this situation, we would want to ask the question: is there a tool that can execute SQL directly on a file? In that case, it would be much more convenient. This article will present some desktop tools for non-professional programmers, allowing them to calculate such files directly in SQL.
Refer to http://c.raqsoft.com/article/1676268429067
##### Post Metadata
- Posted at: há mais de 2 anos
- Net upvotes: 1
# Samples of Processing Big Text File
This article is about applying SPL, explains various operations over big file, illustrates how to make the computation faster using parallel processing, and gives concise sample programs in esProc SPL.
Structured text file processing, such as TXT and CSV files, is common data analysis work. Sometimes the file is too big to be loaded into the memory at one time. You need to load and calculate it in batches and then aggregate the temporary intermediate result sets according to the specific requirements. The whole process is quite different from that of calculating a small file that can be wholly loaded into the memory.
Looking http://c.raqsoft.com/article/1599117027835
##### Post Metadata
- Posted at: há mais de 2 anos
- Net upvotes: 1
# Desktop programming languages suitable for data analyst
Many scenarios need to use programming language to analyze and process the desktop data such as Excel and csv. In these scenarios, simple tasks include querying multiple files of same format through parameter, batch splitting or merging files, processing larger files, etc.; complex tasks include organizing files of irregular format, multi-step calculation, judging the flow of multi branches, calculating by sequence number or location, processing each group of data after grouping, and multi-layer association calculation, etc.
There are many programming languages that meet the above requirements; however, the vast majority of these languages are aimed at professional programmer, such as Java and C#. To use them, we need to understand heavy framework, design the data table object (Data Table or Data Frame), write cumbersome code, and go through painful learning process. In fact, there are some other programming languages, which save us from building heavy framework, come with data table objects, and are simply in coding and easy to learn, and suitable for data analyst to process desktop data. This article will present them one by one.
http://c.raqsoft.com/article/1676962380992
##### Post Metadata
# From Stream to Kotlin to SPL
In terms of structured data calculation outside the database, Stream has taken a step from scratch; Kotlin has slightly enhanced this capability, but the nature of the compiled language makes it impossible to go further; To really solve the problem of structured data computing outside the database, SPL, a professional structured data computing language, is needed. Refer to http://c.raqsoft.com/article/1638847152272
##### Post Metadata
- Posted at: há mais de 2 anos
- Net upvotes: 1
# The impasse of SQL performance optimizing
Many big data calculations are implemented in SQL. When running slowly, we have to optimize SQL, but we often encounter situations that we can do nothing about it. SPL maybe a better choice, refter to http://c.raqsoft.com/article/1640596372891
I'm Jason, the developer of this software, please contact me if you have any questions
##### Post Metadata
- Posted at: há quase 3 anos
- Net upvotes: 1
# General SQL-style Operations in SPL
After we retrieve data from the data source as a table sequence in SPL, we can perform all SQL-style operations, such as filtering, aggregation, inter-column calculations, sorting, grouping & aggregation, top-N, post-grouping top-N, distinct, post-grouping distinct, and join to name a few. In this essay, we take the file data source to explain how to achieve those operations on a SPL table sequence.http://c.raqsoft.com/article/1631170656543
##### Post Metadata
- Posted at: há quase 3 anos
- Net upvotes: 1
# Efficient Data Analysis Engine
esProc can be used for analytical databases and middleware. Its computing power and expression ability far exceed traditional SQL, and it is suitable for online query and offline batch running scenarios. Refer to http://c.raqsoft.com/article/1666484536795
##### Post Metadata
Com mais de 3 milhões de avaliações, podemos fornecer os detalhes específicos que ajudam você a tomar uma decisão informada de compra de software para o seu negócio. Encontrar o produto certo é importante, deixe-nos ajudar.