Example of using SQLAlchemy+Pandas, works with this Cloud SQL as well

from sqlalchemy import create_engine
import pandas as pd

server = '<your-server-hostname>'
port = 1972
namespace = 'USER'
username = 'SQLAdmin'
password = '<YOUR_PASSWORD>'

url = f"iris://{username}:{password}@{server}:{port}/{namespace}"
print(url)
engine = create_engine(url)

df = pd.DataFrame({
    'int': [1, 2, 3, 4, 5],
    'float': [1.1, 2.2, 3.3, 4.4, 5.5],
    'string': ['a', 'b', 'c', 'd', 'e'],
    'datetime': pd.date_range('20130101', periods=5),
    'bool': [True, False, True, False, True]
})

# create a table in IRIS
df.to_sql('iris_table', engine, if_exists='replace', schema='sqlalchemy')

# read the table back from IRIS 
df2 =  pd.read_sql_table('iris_table', engine, schema='sqlalchemy')
# print the dataframe
print(df2)

Is it really the reading file taking so much time or using $piece on the line and setting it to global too?

There are various things here that may slow you, even $increment (best to be replaced by i+1)

You can also split the reading file and set it to global by two parts, and use $sortbegin 

Try to run your code with %SYS.MONLBL started, it will help you understand where it spends more time.

Sorry, but it is the most horrible way to do it.

too old-school, the code has been outdated for many years. Dots syntax in 2023, seriously?

Projections are definitely not a way to solve it and did not get why they were even considered here

The best way to go is using %Studio.SourceControl, there are a lot of examples, and even some are out of the box already.

And most modern way now is to switch from Studio to VSCode, and to local-side development. So, all your classes will always be as files and can be synced to the git repository.

I’m not a fun of using LOAD DATA, it’s not complete, there are some things required to be kept in mind.

now I would recommend to look at dbt tool, which has support for IRIS, and may probably support older versions too, and it works with CSV quite easy, and can create table for you as well. Have a look, and let me know if you have any issues with it. I can fix bugs in IRIS support, if you find any. For IRIS you would need use dbt-iris package