Tuesday, January 29, 2013

Archive files monthly using PowerShell

Problem

My systems creates close 100.000 files every monthly. These files are related to integration or logs and contains only text. After some months the file system will contain more than 1 million files, and also take off quite much storage.

I used Google to get inspiration to create a script to archive they in a structured way. I could probably delete the files, but I want to keep them for some time - in case I need to.

If found some scripts, but none of the did the trick - so I created one myself using PowerShell. The scripts creates and _old folder beneath the folder and archive all files (except .ZIP files) monthly.

The scripts utilitize 7-zip to archive the files.

Solution

<# 
.SYNOPSIS 
   ZIP old files by monthly .ZIP files
.DESCRIPTION
   Creates a .ZIP files in a old\ folder containing files per monthly which is older than first day of the previous month
.NOTES 
   Version: 1.0
   Author : Henrik Næss
#> 

$zipPath = "C:\Program Files (x86)\PortableApps\7-ZipPortable\App\7-Zip64\7z.exe"
$archivemonths = 1

$tempFolder = gc env:temp 
$cutoffdate = (Get-Date -Hour 0 -Minute 0 -Second 0 -Day 1).AddMonths(-$archivemonths)

Function ArchiveFilesByMonth
{
  param(
    [Parameter(Mandatory=$true)][string] $archivefolder,  
    [Parameter(Mandatory=$true)][string] $includePattern
    )

  if(!(Test-Path $archiveFolder))
  {
    Write-Host "Folder $($archiveFolder) doesn't exists"
    return
  }
  Set-Location $archivefolder
  
  $oldFolder = Join-Path $archivefolder "_old\"
  if(!(Test-Path $archiveFolder)) { New-Item $oldFolder -itemtype directory }

  $randomNo = Get-Random
  $filelist = $tempFolder + "filelist$($randomNo).txt"
  $myarray=@()

  gci $archivefolder -recurse -include $includePattern -exclude *.zip | ?{$_.LastWriteTime -lt $cutoffdate} | %{
    #record the month & year of the files in an array
    $temp='' | Select Filenamex, Period
    $temp.Filenamex = ($_ | Resolve-Path -Relative).TrimStart(".\")
    $temp.Period = '{0:MMyyyy}' -f $_.LastWriteTime
    $myarray += $temp
  }
  
  #Get each unique month/year from the array
  foreach($period in ($myarray | Select Period -unique)){
    $zipFileYYYYmm = $oldFolder + ($period.Period).Substring(2, 4) + ($period.Period).Substring(0, 2) + ".zip"
    
    $myarray | ?{$_.Period -eq $period.period} | Select -expand Filenamex | out-file $filelist -encoding ASCII
    $argumentlist = "a -tzip $($zipFileYYYYmm) @$($filelist)"
    
    $result = start-process "$($zipPath)" -argumentlist $argumentlist -NoNewWindow -wait
    
    # If .ZIP created then delete the files
    # can later be changed to move "if using 7-Zip 9.22 beta or later"
    if(Test-Path $zipFileYYYYmm) { gc $filelist | %{gci $_} | %{$_.Delete() } }
  }
  Write-Host "Completed $($archivefolder)"
}

Function ArchivePart1
{
  # exampes
  ArchiveFilesByMonth ("C:\MyFiles\ARCHIVE") AP*
  ArchiveFilesByMonth ("C:\MyFiles\LOGS") AP*
}

ArchivePart1

Wednesday, June 13, 2012

Max Speed with handheld devices isn't accurate!

Do not trust the Max Speed

Finding max speed using a handheld GPS isn't very accurate. I think the most accurate will be using a cycling GPS like Garmin EDGE 800.

Last year I cycled a trip in Mallorca using a normal GPS. I uploaded the track to Strava.com a couple of months ago, and today I re-uploaded the track. Same information, but look at the Max Speed. Changed from 76.3 km/h to 63.8 km/h. Have Strava done some changes?




Conclusion: Do not trust Max Speed, based on handheld GPS.

Friday, May 25, 2012

Find your maximum HR the easy way

Maximum HR

To find your maximum heart rate - the easy way - go test the following http://www.brianmac.co.uk/maxhr.htm

This give you your max HR based on 10 different ways of calculating - e.g. researchers has found out that the normal formula 220 - age is not that accurate for women.

Cycling GPS analyse


Analysis of GPS tracking for cycling

Yesterday I bought a Garmin EDGE 800 with HR and Cadence sensor. Up til now I mainly used my BlackBerry phone to track cycling to and from work. I originally used the GPSLogger app for Blackberry, but lately I switched to Endomondo.  The BlackBerry has been in my backpack.

Sometimes I have been riding alone to work, and other times I been riding in a group of 2-4 people. Now that I get HR data, I expected then energy consumption calculation to be much more accurate.

Summary

Some big differences in calories and elevation gain - see summary and details below. 

I hadn't calibrated the altitude on Edge 800 before my workout, but that shouldn't influence then increase/decrease. The Edge 800 uses barometric to measure height, which is much more accurate than the satellite based.



Strava.com

Tracked with Edge 800

 

Tracked with Blackberry Curve 9000 with Endomondo


Endomondo

Tracked with EDGE 800

Tracked with Blackberry Curve 900


Garmin Connect





Thursday, March 1, 2012

New month - New Milestone

Today it is a new month, 1st of March, and new major milestone at work. Today I launched a new Intranet and Extranet to my company's organization.

The new solution is based on SharePoint 2010 Enterprise Edition. I started the the project in December 2011, and work part time on together with other tasks like a new corporate website plus multiple other projects.

To kick-start the project, I used AutoSPInstaller to install SharePoint. In previous SharePoint projects I have created the PowerShell scripts myself. Fun, to a point - but since I didn't want to spend time in developing and maintaining this now - I decided to go for AutoSPInstaller. A good choice.

The installation went quite smooth. After installation I needed to change the urls, in IIS host headers and SharePoint Central Admin's AAM.

Adding a application server, and moving services from WFE to APP server was more less only stop services on WFE and start them on the APP Server.

After this, it was more or less the standard SharePoint tasks like creating web application, site collections, adding permissions, etc....

My next task is now to configure SCOM, and tune that for SharePoint.


Experienced some problems:

Thursday, February 23, 2012

Is backup and restore for wimps? Not in SharePoint.

Is backup and restore  for wimps? Yes and No.
It is often easy to configure backup, but mostly people doesn't test that it actually test the restore.

Some the day you will be happy that you configured backup and tested the restore. By using DPM in SharePoint this is really easy.

To configure it, use the Wizard in DPM. Be aware that it doesn't include all SharePoint databases, so you may want to include the following databases:
- master
- _MetaDataDB
- _ProfileDB
- _SocialDB
- _SyncDB

To restore items. Start DPM and select Recovery. Then you can browse through the item, and more or less Next -> Next ->..

And then you have it recovered.

References
-Sharepoint 2010 protection in DPM 2010: Part 3
-Data Protection Manager 2010 - Recovering Sharepoint 2010 Items without recovery farm  (YouTube) (skip to 4min 27sec)

 (I haven't misspelled SharePoint, it is the blog author)
·

Tuesday, February 7, 2012

Publishing Multiple SharePoint SSL Applications on Port 443

I am running a SharePoint 2010 installation with two web application: onePortal..mydomain.com and one MySite.domain.com on the same IP. This works well on HTTP using host header and SharePoint alternate access mapping.

There are no way in standard user interface for IIS (7.5) to map two Virtual Web Sites to SSL. The user interface only support assigned one Web Site to SSL.

To fix this, short version:

  1. Buy a wildcard certificate, *.mydomain.com
  2. Install certificate on web server and UAG (in my case)
  3. Assign one the SharePoint web site to https using the default user interface for IIS
  4. For the other(s) start command line in %windir%\system32\inetserv\
  5. appcmd list site (to list existing binding - for you information)
  6. appcmd set site /site.name:"MySite Host" /+bindings.[protocol='https',bindingInformation='*:443:mysite.mydomain.com']
  7. appcmd list site (to list existing binding - for you information)
  8. publish urls in UAG