PowerShell

Using AWS Lambda for PowerShell To Get Data From Microsoft SQL Instance

Standard

I’ve got many PowerShell scripts I’ve written over the years but one of the most useful was to simply get data from a Microsoft SQL server and another to insert or update data. This is an example of modernizing a script I wrote over 5 years ago to now run on serverless computing with AWS Lambda. In addition I can use API Gateway and Lambda to create a serverless interface to a Microsoft SQL instance. I’ve tested this with an Amazon RDS MS SQL standard instance within a VPC, a Lambda function with an ENI within that VPC, and API integration with Lambda. Additionally the security group on the RDS instance will need to allow port 1433 with the source being the Lambda function security group. With this combination of components I can do an HTTPS get request to the API which reads from a MSSQL database. This is also very useful for decoupling, migration, and adopting a microservices architecture.

Continue reading

AWS Lambda for PowerShell Encrypted Environment Variables

Standard

Recently I was developing a AWS Lambda for PowerShell function to read a table from a Microsoft SQL on RDS Instance within a VPC. Using the .NET Core SQL client I need to provide credentials to access the database. For the password it should be stored encrypted at rest, never in plain text, and when passed to the function should be done so as a secure string. We can configure the function to use KMS for encryption at rest. This presents us with two challenges. How do I encrypt a AWS Lambda environment variable and how will my AWS Lambda for PowerShell function securely get the decrypted value in memory as a secure string when the Lambda function is invoked.

Continue reading

Get Detailed Network Connection Information

Standard

This function will list all connections to a Windows computer in PowerShell. While most admins are familiar with Netstat and unless you have Windows 8.1 or Server 2012 R2 there’s no PowerShell equivalent. There’s a .Net method that could be utilized but it doesn’t supply the process identifier (PID) to relate that connection the process using it. Alternatively you create a custom type that uses IP Helper that does have the PID but that’s unnecessarily complicated for what I want. Instead I opted for parsing Netstat’s string output into a format I could use. Additionally I added in DNS lookup for remote addresses, process details from Get-Process, and several parameters to get just what you want.

Continue reading

Mass Merge Data From PowerShell To SQL

Merge-DataTableToSQL
Standard

If you have data that you’re collecting with PowerShell and wan’t to insert new entries or update existing ones in a Database then a SQL MERGE function is what you need. I frequently run scheduled queries to collect information that I want to submit to SQL for reporting purposes. When testing this function I was able to merge a DataTable with 100,000+ rows and 20+ columns very quickly. A unique identifier in the results is used as a primary key in the merger. When there’s a match it updates that row, if it doesn’t exist already it inserts it as a new row. To do this effectively a temporary table is created by first looking up the primary key in the targeted table (if there’s one); then builds the query with all the columns in the source. It also compares the source columns to the target table columns to make sure they correspond for the column mapping. The DataTable is then bulk copied to the temporary table, this becomes the source of the merge to the target table. After which the temporary table is dropped.

Continue reading

Bulk Copy Data From PowerShell to SQL

Copy-DataTableToSQL
Standard

Being able to quickly submit data from PowerShell to SQL is something I often do. Usually I want the results inserted into a SQL table. Being able to do so in bulk without needing a for each insert loop is very helpful and much faster. The easiest way to do this is to use sqlclient.sqlbulkcopy. It requires that the input variable be a DataTable object so if it’s not already use the ConvertTo-DataTable function I wrote earlier. Once you have your data in the DataTable you can use it as the source in this Copy-DataTableToSQL function I wrote.

Continue reading