ArcPyIdlPyIntegration_Exercise

概述 / Overview

本项目的开发目的 / Aim of Development of This Repository

• 整理Python 3.x中的各种与地信, 遥感有关的常用库的使用方法和应用案例. \ To summerize usages and appliances of common libraries in Python 3.x related to GIS/RS.

• 整理arcpy模块(基于ArcGIS Desktop和Python 2.x 32位)在空间数据自动处理中的使用方法和应用案例. \ To summerize usages and appliances of module arcpy (based on ArcGIS Desktop and Python 2.x 32-bit) in automation of spatial data processing.

• 整理idlpy模块(基于ENVI 5.x / IDL 8.x 和 "Python to IDL Bridge")在IDl与Python对接, ENVI遥感数据处理与交互中的使用方法和应用案例. \ To summarize usages and appliances of module idlpy (based on ENVI 5.x / IDL 8.x and "Python to IDL Bridge") in docking IDL with Python, and RS data processing and interaction in ENVI.

• 将arcpy和idlpy嵌入至现有的Python交互式编辑器中, 以供集成使用, 并使用"笔记本式"文档管理代码块. \ To embed arcpy and idlpy into existent interactive editor for Python for intergrated usage. Meanwhile, use notebook-style document for codeblock management.

本项目主要涉及的Python 3.x内建库 / Built-in Python 3.x Libraries Mentioned in This Repository

• 基本数学运算库. \ Libraries for fundermental mathematical operation.
  • math, cmath
  • decimal, fractions
  • statistics, random
• 字符串和文本文件处理相关库. \ Libraries for processing of character strings and text files.
  • string, re
  • sys, os, io
• 与GIS/RS有关的可交换文本文件导入导出, 如CSV, JSON/GeoJSON, XML/KML. \ Libraries for import and export of GIS/RS-related exchangable textual files, e.g. CSV, JSON/GeoJSON, XML/KML
  • csv, json, xml
• Sqlite数据库读写. \ Libraries for reading and writing Sqlite database.
  • sqlite3

本项目主要涉及的第三方Python库 / Third-Party Python Libraries Mentioned in This Repository

• 数学建模, 数据挖掘, 通用科学数据计算相关库. \ Libraries for mathematical modeling, data mining and general scientific data computation.
  • numpy
  • scipy
  • pandas
• ArcGIS Desktop的Python 2.x接口. \ Python 2.x interface of ArcGIS Desktop.
  • arcpy
• IDL / ENVI的Python 2.x接口 (Python to IDL Bridge). \ Python 2.x interface of IDL / ENVI (Python to IDL Bridge).
  • idlpy

运行环境信息 / Information of Environment for Execution

系统和程序包环境 / Environment of System and Packages

|项目
Item|值
Value| |:-:|:-| |处理器架构
Architecture of processor|Intel x86-64| |操作系统内核
OS kernel version|Windows NT 6.1.7601| |Conda管理器版本
Conda package manager version|conda 4.7.12 (python 3.7.4, requests 2.22.0)| |Anaconda发行版
Anaconda system release version|anaconda 2019.10| |ArcGIS发行版
ArcGIS release version|ArcGIS Desktop 10.5| |ENVI发行版
ENVI release version|ENVI 5.3|

配置arcpy, idlpy集成环境 / Configure Integrated Environment of arcpy and idlpy

本项目通过anaconda环境管理器, 创建一个名为arcpy_idlpy_x32的子环境, 集成arcpy和idlpy以供二次开发使用. 上述二次开发接口对接的软件具有以下特征: \ This repository creates an environment named arcpy_idlpy_x32 via anaconda environment manager, which intergrates arcpy and idlpy for secondary development. Softwares docked by interfaces above contains features as follow:

• ArcGIS Desktop 10.x主软件可在32位或64位系统下运行, 但附带的python解释器为python 2.x版本, 仅包含32位版本. \ The master product of ArcGIS Desktop 10.x can run on 32-bit or 64-bit systems, while its python interpreter is python 2.x, containing 32-bit release only.

• ENVI 5.3/IDL 8.5的python接口 (Python to IDL Bridge) 支持python 2.x/3.x, 同时提供32位和64位版本. \ The python interface of ENVI 5.3/IDL 8.5 (Python to IDL Bridge) supports python 2.x/3.x, providing both 32-bit and 64-bit releases.

因此, 需要创建一个相对独立的, 32位的python 2.x环境, 用于运行解释器并调用相关程序包. \ Therefore, creation of a standlone 32-bit python 2.x environment to run interpreter and invoke relative packages is required.

[!IMPORTANT]

在执行后续操作前, 可能需要关闭反病毒软件和第三方防火墙软件, 因为上述软件可能会阻止anaconda从镜像源下载程序包.

Before execution of following operations, developers are supposed to disable anti-virus softwares and third-party firewalls, which may block anaconda from downloading packages from mirror websites.

确定ArcGIS及其python解释器安装路径 / Verify Installation Path of ArcGIS and Its python Interpreter

ArcGIS Desktop 10.x在Windows系统中安装时, 安装向导会在准备阶段分两步指定ArcGIS主软件, 及其python解释器的安装目录, 这两个目录可能不同. 因此, 需要将二者区分开来. 需要确定 ~~ArcGIS的~~python解释器所在目录以完成后续操作. \ During installation of ArcGIS Desktop 10.x on Windows, installation wizard will assign directories of ArcGIS master product, and its python interpreter, in two steps separately. Directories above may be different and need to be distinguished. To archive further operations, developers are supposed to verify directory in which the python interpreter of ArcGIS locates.

在Anaconda prompt命令行中, 将 ~~ArcGIS的~~python解释器所在目录设置为临时变量. \ In Anaconda prompt, set a temporary variable to store the path of the python interpreter of ArcGIS.

[!NOTE] 本文档中以绝对路径(以盘符开头)表示的安装路径仅供参考. 请以开发用机上python解释器的实际安装路径为准.

In current document, installation paths represented as ABSOLUTE paths (beginning with drive letter) are FOR REFERENCE ONLY. Please consult installation path of python interpreter de facto.

bash set pyarcgis=D:\python27\ArcGIS10.5

[!IMPORTANT] 临时变量的值仅在当前命令行会话中有效, 命令行关闭后将被清理或者复原.

执行上述命令后, 请勿关闭命令行界面, 以便在后续命令中继续使用临时变量. 后续操作均在同一控制台中执行.

Values of temporary variables are valid in CURRENT CLI SESSION ONLY and will be cleared or restored after closing the console.

After excuting command above, do NOT close CLI, so that variables can be reused in following commands. Further operations will be executed in THE SAME CONSOLE.

确定IDL安装路径 / Verify Installation Path of IDL

在Anaconda prompt命令行中, 将IDL安装目录设置为临时变量. \ In Anaconda prompt, set a temporary variable to store the installation path of IDL interpreter.

bash set idl=D:\Exelis\IDL85

[!NOTE] 上述路径不是IDL的解释器(idl.exe或者idlrt.exe)所在的路径.

路径下包含bin, examples, external, help, lib, resource等子目录.

Path above is NOT the path where the interpreters of IDL (idl.exe or idlrt.exe) locate.

This path contains subdirectories including bin, examples, external, help, lib, resource.

确定anaconda安装路径 / Verify Installation Path of anaconda

在Anaconda prompt命令行中, 将anaconda安装目录设置为临时变量. \ In Anaconda prompt, set a temporary variable to store the installation path of anaconda.

bash set anacon=D:\Anaconda3

建立并初始化环境 / Create and Initiallize Environment

1. 为建立新环境初始化anaconda系统参数. \ Initialize parameters of anaconda for creation of new environment.

bash set CONDA_FORCE_32BIT=1 conda clean -i

1. 建立并进入新环境arcpy_idlpy_x32. \ Create and enter new environment arcpy_idlpy_x32.

bash conda create -n arcpy_idlpy_x32 python==2.7.12 pip==8.1.1 wheel==0.29.0 -y conda activate arcpy_idlpy_x32

1. 在新环境中配置与ipython有关的包. \ Configure packages related to ipython in new environment.

bash conda install notebook ipykernel numpy==1.9.3 six==1.10.0 tornado==4.5.3 -y

1. 将ArcGIS的python解释器使用的部分包链接到新环境下. Link several packages used by python interpreter of ArcGIS to new environment.

bash chdir /D %anacon%\envs\arcpy_idlpy_x32\Lib\site-packages mklink /D scipy %pyarcgis%\Lib\site-packages\scipy mklink /D pandas %pyarcgis%\Lib\site-packages\pandas mklink /D matplotlib %pyarcgis%\Lib\site-packages\matplotlib mklink /D pytz %pyarcgis%\Lib\site-packages\pytz mklink /D dateutil %pyarcgis%\Lib\site-packages\dateutil mklink pyparsing.py %pyarcgis%\Lib\site-packages\pyparsing.py mklink pyparsing.pyc %pyarcgis%\Lib\site-packages\pyparsing.pycpytz mklink cycler.py %pyarcgis%\Lib\site-packages\cycler.py mklink cycler.pyc %pyarcgis%\Lib\site-packages\cycler.pyc

1. 将arcpy的路径配置文件链接到新环境下. \ Link path configuration file of arcpy to new environment.

bash mklink Desktop10.5.pth %pyarcgis%\Lib\site-packages\Desktop10.5.pth

1. 在新环境下建立idlpy的路径配置文件. \ Create path configuration file of idlpy in new environment.

bash echo %idl%\bin\bin.x86 >IDL8.5.pth echo %idl%\lib\bridges >>IDL8.5.pth

1. 将新环境注册为ipython内核, 以便与base环境下的Jupyter notebook一同使用. \ Register new environment as an ipython kernel for use with Jupyter notebook in base environment.

bash pip install backports.functools_lru_cache ipython kernelspec install-self

[!NOTE] 如果内核注册不成功(发生报错), 一般是因为pip未能向环境内成功安装上述库, 如遇上述现象, 需要再通过conda安装该库.

The faliure of kernel registration is generally caused by faliure of installation of library above via pip, In case of phenomenon above, it is required to install this library via conda again.

bash conda install backports.functools_lru_cache

1. 完成新环境配置. \ Finish configuration of new environment.

conda deactivate set CONDA_FORCE_32BIT=.

相关程序包的使用 / Usage of Relative Packages

使用集成环境中的部分数据分析和科学计算相关库 / Use Libraries for Data Analysis and Scientific Computation in Integrated Environment

对应的import语句需要重复调用多次, 才可正常导入模块. 需要在代码开头声明以下语句: \ Only by invoking corresponding import statements repeatedly can modules be imported properly. It is required to declare following statements at the beginning of your code:

python module_imported = False; while not(module_imported): try: import numpy, scipy, pandas; except SyntaxError: continue; else: break;

使用arcpy和idlpy / Use arcpy and idlpy

直接使用一次import语句即可. \ Directly use import statements once.

python import arcpy, idlpy;