What is Web Scraping:

In simple terms, Web scraping, web harvesting, or web data extraction is an automated process of collecting large data(unstructured) from websites. The user can extract all the data on particular sites or the specific data as per the requirement. The data collected can be stored in a structured format for further analysis.

17 12
9 1.3K

In the previous parts (1, 2) we talked about globals as trees. In this article, we will look at them as sparse arrays.

A sparse array - is a type of array where most values assume an identical value.

In practice, you will often see sparse arrays so huge that there is no point in occupying memory with identical elements. Therefore, it makes sense to organize sparse arrays in such a way that memory is not wasted on storing duplicate values.

In some programming languages, sparse arrays are part of the language - for example, in J, MATLAB. In other languages, there are special libraries that let you use them. For C++, those would be Eigen and the like.

Globals are good candidates for implementing sparse arrays for the following reasons:

8 3
1 1.3K

Imagine that your .NET project uses the Caché DBMS and you need a fully-functional and reliable authorization system. Writing such a system from scratch would not make much sense, and you will clearly want to use something that already exists in .NET, e.g. ASP.NET Identity. By default, however, this framework supports only its native DBMS – MS SQL. Our task was to create an adaptor that would let us quickly and easily port Identity to the InterSystems Caché DBMS. This work resulted in creation of the ASP.NET Identity Caché Provider.

4 1
0 1.3K

cAdvisor (short for container Advisor) analyzes and exposes resource usage and performance data from running containers. cAdvisor exposes Prometheus metrics out of the box.

https://prometheus.io/docs/guides/cadvisor/

Prometheus is integrated in SAM. This makes it possible to leverage the cAdvisor metrics and expose them via Prometheus and Grafana.

5 2
0 1.3K

Pouring The Coffee: Creating and scheduling a task

Don't you wish a fresh, hot cup of coffee could be waiting for you right when you get into the office? Let's automate that!

Cache and IRIS come with a built-in Task Manager, which should have a familiar feel to those used to using the Windows task scheduler or using cron on Linux. Your user account will need access to the %Admin_Task resource to use it, and you can access it in the management portal under System Operation -> Task Manager. When first installed, there are roughly 20 types of task that you can schedule.

10 7
5 1.3K

1. Scope and Objective:

Recently we supported a few NHS cases that required TIE (Trust Integration Engine) integration with the PKB service. Hence this article is meant to be a 10-minute quick guide to describe a demo solution (simple configurations and end-2-end implementation steps) for Health Connect (Ensemble) Integration with PKB (Patient-Knows-Best) service.

5 0
1 1.3K

System Monitor is a flexible and highly configurable tool supplied with Caché (Ensemble, HealthShare), which collects the essential metrics of the operating system and Caché itself. System Monitor also notifies administrators about issues with Caché and the operating system, when one or several parameters reach the admin-defined thresholds.

1 2
2 1.3K

This is the second article in a series on iKnow demo applications, showcasing how the concepts and context provided through iKnow's unique bottom-up approach can be used to implement relevant use cases and help users be more productive in their daily tasks. Last week's article discussed the Knowledge Portal, a straightforward tool to browse iKnow indexing results.

This week, we'll look into the Set Analysis demo, a slightly more advanced application where you'll be using the concepts identified by iKnow to organize your content into sets of documents. The original version of this demo was developed by Danny Wijnschenk & Alain Houf for an academy session at GS2015, but the app has evolved significantly since then.

5 10
0 1.3K

Continuing on with providing some examples of various storage technologies and their performance profiles, this time we looked at the growing trend of leveraging internal commodity-based server storage, specifically the new HPE Cloudline 3150 Gen10 AMD processor-based single socket servers with two 3.2TB Samsung PM1725a NVMe drives.

4 2
0 1.3K
Article
· Jul 27, 2018 4m read
Load a ML model into InterSystems IRIS

Hi all. Today we are going to upload a ML model into IRIS Manager and test it.

Note: I have done the following on Ubuntu 18.04, Apache Zeppelin 0.8.0, Python 3.6.5.

Introduction

These days many available different tools for Data Mining enable you to develop predictive models and analyze the data you have with unprecedented ease. InterSystems IRIS Data Platform provide a stable foundation for your big data and fast data applications, providing interoperability with modern DataMining tools.

6 2
2 1.3K
This is the second part of my long post about package managers in operating systems and language distributions. Now, hopefully, we have managed to convince you that convenient package manager and rich 3rd party code repository is one key factor in establishing of a vibrant and fast growing ecosystem. (Another possible reason for ecosystem success is the consistent language design, but it will be topic for another day.)
6 23
1 1.3K
Article
· Nov 23, 2017 12m read
Where is my global stored?

It's well-known that namespace global mapping helps us to write code independent on database storage details (Caché instance name, directory path). But sometimes we can face problems accessing an unsubscripted global which has subscript level mapping (SLM) defined. Most of such cases are evident and associated with administrative tasks that should be done on database level, but some of them can confuse even an experienced developer. Just to start:

4 8
0 1.3K

or "Things are going to break"

We left our application over the weekend, secure in the knowledge that it was returning data from our primary persistent class, User.Widget. However, Widgets Direct are the premier supplier of both Widgets AND Widget Accessories, so we should really start working on adding these Accessories to our application.

2 3
0 1.3K

Advent of Code is a series of 25 small programming challenges, it's an ideal way for beginners to start learning a computer language, and for advanced people to sharpen their programming skills.

There are small and bigger puzzles, which you can solve typically in half an hour to a few hours. (Looking at the leaderboard, the top aces can do them in less than 10 minutes.)

Advent of Code is created by Eric Wastl, you can find all info on https://adventofcode.com/.

7 0
1 1.3K

Introduction

Nowadays, there is a lot of applications that are using Open Authorization framework (OAuth) to access resources from all kinds of services in a secure, reliable and efficient manner. InterSystems IRIS is already compatible with OAuth 2.0 framework, in fact, there is a great article in the community regarding OAuth 2.0 and InterSystems IRIS in the following link here.

5 1
0 1.3K
Article
· Jan 16, 2017 15m read
Part I – Thoughts about package manager

Have you ever thought what could be a reason why some development environment (database, language) would eventually become popular? What part of this popularity could be explain as language quality? What by new and idioms approaches introduced by early language adopters? What is due to healthy ecosystem collaboration? What is due to some marketing genius?

10 2
2 1.3K
Article
· Jun 30, 2022 8m read
What's new in Angular 14

Hi! My name is Sergei Sarkisian and I’m creating Angular frontend for more than 7 years working in InterSystems. As the Angular is very popular framework, our developers, customers and partners often choose it as part of the stack for their applications.

I would like to start series of articles which will cover different aspects of Angular: concepts, how-to, best practices, advanced topics and more. This series will target people who already familiar with Angular and wouldn’t cover basic concepts. As I’m in the process of building articles roadmap, I would like to begin with highlighting some important features in most recent Angular release.

3 1
0 1.3K

"Telegram" is a well-known instant messenger, which provides an API for creating bots. The features of this API allow you to create bots with a wide range of functionality including receiving payments.
With the help of the telegram bot, I solved a simple task - sending Alerts from Ensemble to Telegram.

Advantages: Alerts come to the mobile phone, a notification appears, so there is no need to install any additional applications (in contrast to the solution https://community.intersystems.com/post/sending-alerts-mobile-phone-using-pushover- httpoutboundadapter).

The bot can be extended by adding new commands, for example, for managing the Production or for solving other tasks in Iris, Ensemble or Cache.

14 0
5 1.3K

One of the topics that comes up often when managing Ensemble productions is disk space:

The database (the CACHE.DAT file) grows in a rate that was unexpected; or the Journal files build up at a fast pace; or the database grows continuously though the system has a scheduled purge of the Ensemble runtime data.

It would have been better if these kind of phenomena would have been observed and accounted for yet at the development and testing stage rather than on a live system.

For this purpose I created a basic framework that could aid in this task.

4 7
2 1.3K

In this series of articles, I'd like to present and discuss several possible approaches toward software development with InterSystems technologies and GitLab. I will cover such topics as:

  • Git 101
  • Git flow (development process)
  • GitLab installation
  • GitLab Workflow
  • Continuous Delivery
  • GitLab installation and configuration
  • GitLab CI/CD
  • Why containers?
  • GitLab CI/CD using containers

In the first article, we covered Git basics, why a high-level understanding of Git concepts is important for modern software development, and how Git can be used to develop software.

In the second article, we covered GitLab Workflow - a complete software life cycle process and Continuous Delivery.

In the third article, we covered GitLab installation and configuration and connecting your environments to GitLab

In the fourth article, we wrote a CD configuration.

In this article, let's talk about containers and how (and why) they can be used.

4 0
1 1.3K

The following steps show you how to display a sample list of metrics available from the /api/monitor service.

In the last post, I gave an overview of the service that exposes IRIS metrics in Prometheus format. The post shows how to set up and run IRIS preview release 2019.4 in a container and then list the metrics.


This post assumes you have Docker installed. If not, go and do that now for your platform :)

17 10
5 1.3K

Something that shot up the popularity stakes last week was this article on a very interesting initiative: RealWorld:

https://medium.com/@ericsimons/introducing-realworld-6016654d36b5

I decided it would be a good idea to use this as a way of creating an exemplar implementation of a RESTful back-end using QEWD against their published API (https://github.com/gothinkster/realworld/tree/master/api)

The results are here:

4 14
0 1.3K