Wednesday, January 27, 2021

How to programmatically fetch an NPM package's latest version with Node JS

We have several front-end modules, all of which utilize a shared enterprise UI/UX library.  We needed a way to keep this shared library's dependency version in sync, therefore, we needed a way to programmatically pull the latest library package version from our local registry.

I am not a big fan of utilizing dependencies to do something which we can do ourselves with minimal effort.  Therefore, I put together a quick function (with some help from the npmview project) which allows us to pull the latest package version using the undocumented npm API.  Note that, because it's using the undocumented API that it could break at any time.  I've tested the function with npm version 6.14.10 & 6.14.11 and it seems to work fine.

const npm = require('npm');

function getLatestPackageVersion() {
  return new Promise(function (resolve, reject) {
    npm.load({ loglevel: 'silent' }, function (err) {
      if (err) {
        reject(`Failed to initialize NPM: ${err.toString()}`);

        return;
      }

      // A hack to shut the NPM registry the hell up -- taken from npmview -- may not always work
      if (npm && npm.registry && npm.registry.log && npm.registry.log.level) {
        npm.registry.log.level = 'silent';
      }

      // Replace lodash with the package you'd like to fetch the latest version of
      npm.commands.show(['lodash', 'version'], true, function (err, data) {
        if (err) {
          reject(`Failed to get latest package version: ${err.toString()}`);

          return;
        }

        // Resolve with the latest package version
        resolve(Object.keys(data)[0]);
      });
    });
  });
}

async function main() {
  try {
    console.log('Latest version: ', await getLatestPackageVersion());
  } catch (error) {
    console.error('Error: ', error);
  }
}

main();

Monday, October 19, 2020

Cinnamon boots to black screen with cursor, then restarts in a loop, the logs reference systemd / dbus timeout

I've been running Linux Mint now for ~10 years after earlier forays into Gentoo, Slackware & Arch.  Back when I was a teenager and I didn't have the cash to purchase new hardware and needed to squeeze every inch of performance out of my machines, distributions which gave me fine-grained control of what was running in the kernel & user space not only gave me this last bit of performance, but also taught me a lot about running Linux & GNU in general.

Since moving over to Linux Mint, things have usually "just worked" without the need for me to actively manage my OS.  Unfortunately, computers are computers and sometimes things break.  I don't have to reboot often, though I do like to keep my machine up-to-date.  After keeping my machine up for a month or two, I noticed that suddenly I was experiencing catastrophic failure--the network was working, but Network Manager was not.  I was unable to perform many tasks on the machine, therefore, I rebooted.

The machine appeared to reboot fine, albeit taking ~5 minutes longer than normal, though once I entered into X is when I noticed that lightdm was not coming up--I would see a pointer on a black screen before X cycled into a loop, exiting to a blinking cursor and then back to the black screen with pointer.

Quickly dropping into my terminal & killing lightdm, I began reviewing the lightdm logs to see exactly where the problem lay.  I saw several lines like so:

WARNING: Failed to get list of logind seats: GDBus.Error:org.freedesktop.DBus.Error.TimedOut: Failed to activate service 'org.freedesktop.login1': timed out (service_start_timeout=25000ms)

And so:

WARNING: Error getting user list from org.freedesktop.Accounts: GDBus.Error:org.freedesktop.DBus.Error.TimedOut: Failed to activate service 'org.freedesktop.Accounts': timed out (service_start_timeout=25000ms)

Thinking possibly that a package broke my system, I proceeded to perform an apt update / apt upgrade, though while running apt I also was seeing the following suspicious error:

Error: Timeout was reached
Reading package lists... Done

Digging into my syslog also showed multiple service start failures, all pointing back to a DBus timeout:

Oct 18 16:13:14 hostname accounts-daemon[1173]: error getting polkit authority: Error initializing authority: Error calling StartServiceByName for org.freedesktop.PolicyKit1: GDBus.Error:org.freedesktop.DBus.Error.TimedOut: Failed to activate service 'org.freedesktop.PolicyKit1': timed out (service_start_timeout=25000ms)
Oct 18 16:13:14 hostname udisksd[1164]: Error initializing polkit authority: Error initializing authority: Error calling StartServiceByName for org.freedesktop.PolicyKit1: GDBus.Error:org.freedesktop.DBus.Error.TimedOut: Failed to activate service 'org.freedesktop.PolicyKit1': timed out (service_start_timeout=25000ms) (g-dbus-error-quark, 20)
Oct 18 16:13:14 hostname ModemManager[1172]: <warn>  failed to create PolicyKit authority: 'Error initializing authority: Error calling StartServiceByName for org.freedesktop.PolicyKit1: GDBus.Error:org.freedesktop.DBus.Error.TimedOut: Failed to activate service 'org.freedesktop.PolicyKit1': timed out (service_start_timeout=25000ms)'

After doing some research I came across some posts regarding the transition away from /var/run/dbus as a standalone folder, and rather a symlink to /run/dbus:
 
Clearly, there was something wrong with my /var/run configuration--after checking, it appears that some package had replaced /var/run with a real folder--instead of a symlink to /run.

After re-creating the symlink with the following command and rebooting, everything was back to normal 🎉:

ln -s /run /var/run

Note: After writing this blog post, I realized that the root cause of the issue & the solution were clearly documented on the Linux Mint Releases & Announcements form here: https://forums.linuxmint.com/viewtopic.php?t=331605

Tuesday, September 29, 2020

The npm pack command works fine manually, but refuses to work when utilizing a Gradle wrapper

It's been a while since my lost post and since then I've moved far into the world of JavaScript development (primarily the React/Angular space).  I just came across something I spent entirely too long on, which I definitely want to share in case someone else makes the same mistake I did.

We recently began migrating our NPM pipelines to a Jenkins / Gradle based system.  With the introduction of Gradle, we needed to create an NPM wrapper script which would be able to properly produce our library artifact (via npm pack), via the same steps are our current build system performed, and naturally in the same order.

Knowing this was the case, I went ahead and determined the steps which were currently called so I could test them manually and ensured they also produced the appropriate artifact.  These steps were actually extremely simple & standard for an NPM library:

  • npm install
  • npm run build
  • npm pack
  • npm publish artifact-1.0.45.tgz --registry https://our-internal-registry.local/artifactory/api/npm/repo-artifact-npm

Straight forward enough, I went ahead & developed a Gradle script which would call the first three steps in the appropriate order.  Though the plugin has since been deprecated, I was already familiar with the com.moowork.node Gradle plugin.  Knowing I didn't need much to spin up a simple script, I went ahead & developed the following:

apply plugin: "com.moowork.node"

defaultTasks 'artifactNpmPack'

node {
  download = false
}

task build {
  dependsOn("artifactNpmPack")
}

task artifactNpmInstall(type: NpmTask) {
  args = ['install']
}

task artifactNpmBuild(type: NpmTask) {
  args = ['run', 'build']
}

task artifactNpmPack(type: NpmTask) {
  args = ['pack']
}


This script should, when ran as gradle build essentially run the same commands as the manual ones above with no problems.

Unfortunately, I spent the next 12 hours delving into AV issues, file / folder auditing, ProcMon, and finally the NPM pack source code before determining the silly overlooked cause of my issue.  What was my issue, one would ask?  Well, see below (lifted from a Stack Overflow question as it's the exact same issue):

npm ERR! path c:\Temp\npm-20936-b98f84c8\tmp\fromDir-02dd5394\package.tgz
npm ERR! code EPERM
npm ERR! errno -4048
npm ERR! syscall unlink
npm ERR! Error: EPERM: operation not permitted, unlink 'c:\Temp\npm-20936-b98f84c8\tmp\fromDir-02dd5394\package.tgz'
npm ERR!     at Error (native)
npm ERR!  { Error: EPERM: operation not permitted, unlink 'c:\Temp\npm-20936-b98f84c8\tmp\fromDir-02dd5394\package.tgz'
npm ERR!     at Error (native)
npm ERR!   cause:
npm ERR!    { Error: EPERM: operation not permitted, unlink 'c:\Temp\npm-20936-b98f84c8\tmp\fromDir-02dd5394\package.tgz'
npm ERR!        at Error (native)
npm ERR!      errno: -4048,
npm ERR!      code: 'EPERM',
npm ERR!      syscall: 'unlink',
npm ERR!      path: 'c:\\Temp\\npm-20936-b98f84c8\\tmp\\fromDir-02dd5394\\package.tgz' },
npm ERR!   isOperational: true,
npm ERR!   stack: 'Error: EPERM: operation not permitted, unlink \'c:\\Temp\\npm-20936-b98f84c8\\tmp\\fromDir-02dd5394\\package.tgz\'\n    at Error (native)',
npm ERR!   errno: -4048,
npm ERR!   code: 'EPERM',
npm ERR!   syscall: 'unlink',
npm ERR!   path: 'c:\\Temp\\npm-20936-b98f84c8\\tmp\\fromDir-02dd5394\\package.tgz' }
npm ERR!
npm ERR! Please try running this command again as root/Administrator.

Unfortunately, the solutions which worked for others here did not work for me.  After losing my mind for almost 12 hours, I discovered that I was attempting to add the "./gradle/buildOutputCleanup/buildOutputClean.lock" file into the package when running "npm pack" -- this makes sense as I didn't previously have a ".gradle" directory, and somehow it slipped my mind that it must be added to .gitignore.

The solution was simply adding the following to .gitignore, and my build was successful:

# Gradle
# .gradle/ 
 

Wednesday, November 15, 2017

Quickly find uninstall GUID of product in Programs and Features

Ever get tired of digging through the registry HKLM\Software\Microsoft\Windows\CurrentVersion\Uninstall and HKLM\Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall keys searching for a product?  Luckily, there's a much faster way to find the GUID, simply run the following command--in this case, we are searching for the Symantec uninstall string:

wmic product get | findstr Symantec

Though the formatting of the result is off, it's still quite easy to locate the GUID and proceed to uninstall the product like so:

msiexec /x {guid}

Thursday, November 9, 2017

Skype for Business / Lync 2013 Outlook Properties Dialog Implementation

Since the advent of Office 2013, I've been increasingly frustrated with the new Outlook contact cards, and the round-about way needed to access the legacy GAL properties dialog as shown below:


After a ton of research, I discovered it's possible to add custom context menus within the Lync 2013 / Skype for Business client, as per this page from Microsoft.  From here, I needed to discover the API necessary to actually launch the legacy GAL properties dialog box.  Doing a bit of research, I simply could not find an easy way to do this, the only thing I was coming across was the following registry key, which just launched the GAL properties box from within Outlook:

HKEY_CURRENT_USER\Software\Microsoft\Office\16.0\common\contactcard
DWORD: TurnOnLegacyGALDialog
Value: 1 (enable)

Nearly giving up after hours of digging through the API and searching, I was provided some help via Dmitry Streblechenko on StackOverflow (thank you, Dmitry!).

In the end, I ended up with a simple VBS and registry key:

LaunchGALProperties.vbs
If WScript.Arguments.Count = 2 Then
                Dim objOutlook : Set objOutlook = CreateObject("Outlook.Application")
                Dim objNamespace : Set objNamespace = objOutlook.GetNamespace("MAPI")
               
                objNamespace.CreateRecipient(WScript.Arguments(1)).AddressEntry.Details()
               
                Set objOutlook = Nothing
                Set objNamespace = Nothing
End If


LaunchGALProperties.reg
Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\Office\16.0\Lync\SessionManager\Apps\{4C13AA00-01F0-428C-B582-38E4F726D97E}]
"Name"="Outlook Properties"
"ApplicationType"=dword:00000000
"SessionType"=dword:00000000
"ExtensibleMenu"="ConversationWindowRightClick;MainWindowRightClick;MainWindowActions;ConversationWindowActions;ContactCardMenu"
"ApplicationInstallPath"="C:\\\\windows\\\\System32\\\\wscript.exe"
"Path"="C:\\\\windows\\\\System32\\\\wscript.exe C:\\\\temp\\\\LaunchGALProp\\\\LaunchGALProperties.vbs \"%user-id%\" \"%contact-id%\""
Boom!  After restarting Skype for Business, I was treated with the following view:


It's quite unfortunate that Microsoft removed this functionality within the newer versions of the Lync/Skype clients, but at least with a little bit of elbow grease I was able to recover it...for now.

UPDATE 04/29/2019:

It appears that the "wscript" command is no longer working (for some reason).  I was able to workaround this by creating a C# project with the following source code:

Program.cs
using System;

namespace LaunchGALProps {
class Program
{
static void Main(string[] args) {
if(args.Length == 2)
{
dynamic objApp;
dynamic objNamespace;

objApp = Activator.CreateInstance(Type.GetTypeFromProgID("Outlook.Application"));
objNamespace = objApp.GetNamespace("MAPI");

objNamespace.CreateRecipient(args[1]).AddressEntry.Details();
}
}
}
}

If you do not have Visual Studio installed, but you DO have .NET v4+ installed, you can compile the application like so:

C:\Windows\Microsoft.Net\v4.0.30319\csc.exe /out:LaunchGALProps.exe Program.cs

After my program was compiled, I was able to update my registry key values for ApplicationInstallPath and Path -- once this was done everything was working again.


Friday, July 14, 2017

Find All Servers in OU Utilizing dsquery

Recently at work I needed to pull all servers from a specific OU and Get-ADComputer was not working for some reason.  I didn't feel like taking the time to figure it out so I just reverted back to handy dsquery:

dsquery * OU=Test,DC=contoso,DC=local -scope subtree -limit 999999999 -filter "(&(objectClass=Computer)(objectCategory=Computer))" -attr sAMAccountName operatingSystem

Wednesday, July 12, 2017

JavaScript Convert String to Bytes Function

Just wanted to post a quick JavaScript convert string to bytes function just for myself, but I figured someone else may find it useful:

function convertToBytes(str) {
     var bytes = [];
     
     for(var i = 0; i < str.length; ++i) {
          var charCode = str.charCodeAt(i);
          bytes = bytes.concat([charCode & 0xff, charCode / 256 >>>             0]);
     }

     return bytes;
}

And my slightly improved, ES6 compatible version:

function convertToBytes(str) {
     var bytes = [];
     
     Array.from(str).forEach((char) => {
          var charCode = char.charCodeAt(char);
          bytes = bytes.concat([charCode & 0xff, charCode / 256 >>>             0]);
     });

     return bytes;
}

That funky bytes concatenation line simply enables the function to work with Unicode characters.

All credit goes out to BrunoLM at StackOverflow.

Monday, December 14, 2015

Microsoft KB 3109103 Breaks WebSockets

Recently my gulp live reload broke like so:

ws://x.x.x.x:8080/LiveReload' failed: Error during WebSocket handshake: Unexpected response code: 200

Even though nothing changed, I noted patches had been installed over the weekend.  After looking through which files were updated, I noticed that KB 3109103 patched Wshrm.dll.  I was unable to find much useful information on this DLL, but after looking into the description it appeared to have a lot to do with sockets:  Windows Sockets Helper DLL for PGM.

After removing the update & rebooting my web sockets started working again.

Wednesday, December 2, 2015

MSI Installation Failure - 1328. Error applying patch.

Wow this one killed me.  After attempting to determine why I simply could not install Adobe DC with our package from enterprise (though only on certain machines) I really had to dig into this one.  The package would first fail with a generic:

Product: Adobe Acrobat DC - Update 'Adobe Acrobat DC (15.006.30060)' could not be installed.  Error code 1603.

Knowing that this is a very generic error code, I began digging through a "l*v" generated log to determine the true source of the failure.

MSI (s) (84:88) [14:36:07:292]: Product: Adobe Acrobat DC -- Error 1328.Error applying patch to file C:\Config.Msi\PT2AFE.tmp.  It has probably been updated by other means, and can no longer be modified by this patch.  For more information contact your patch vendor.  System Error: -1072807676

Error 1328.Error applying patch to file C:\Config.Msi\PT2AFE.tmp.  It has probably been updated by other means, and can no longer be modified by this patch.  For more information contact your patch vendor.  System Error: -1072807676

Aha, so it appeared that for some reason the MSP packaged with the MSI was failing to be applied.  Scrolling up through the log I was able to identify the actual failing file itself:

MSI (s) (84:88) [14:36:07:292]: Executing op: CacheBaselineFile(Baseline=0,FileKey=adobe_caps.dll,FilePath=C:\Program Files (x86)\Adobe\Acrobat 2015\Acrobat\adobe_caps.dll,,Existing=0)
MSI (s) (84:88) [14:36:07:292]: Executing op: PatchApply(PatchName=adobe_caps.dll,TargetName=C:\Program Files (x86)\Adobe\Acrobat 2015\Acrobat\adobe_caps.dll,PatchSize=81904,TargetSize=552632,PerTick=0,,FileAttributes=16384,PatchAttributes=0,CheckCRC=0)
MSI (s) (84:88) [14:36:07:292]: Patch for file 'C:\Program Files (x86)\Adobe\Acrobat 2015\Acrobat\adobe_caps.dll' is redirected to patch 'C:\Config.Msi\PT2AFE.tmp' instead.

At this point I needed to dig deeper--I know that the package installed perfectly on other machines, so why not this one?  Firing up Procmon, I was able to see that msiexec.exe was hitting the "adobe_caps.dll" file here:

C:\Windows\Installer\$PatchCache$\Managed\68AB67CA3301FFFF7706E0F060571500\15.6.30033

Being unfamiliar with the "$PatchCache$" folder, I did a little research and found that files can be cached here (and only cached ONCE) for any MSP file installed on your machine.  Since the installer was being pulled across the network, it looks like a network hiccup occurred and a corrupt version of the "adobe_caps.dll" (showing as 0 KB) was stored in cache.

To resolve the error, all that needed to be done was to remove the entire "68AB67CA3301FFFF7706E0F060571500" folder & reinstall!  Bam!!

Thursday, November 19, 2015

Getting NW.js (node-webkit) v0.12.3 to work with Edge.js

This is something I needed to get functioning for work.  Luckily it wasn't so difficult--the hardest part was attempting to figure out why after I compiled Edge.js properly for NW.js v0.12.3 why it wouldn't actually attempt to load the native "edge.node" module.

First things first, we need to make sure the following is installed on your machine:

Windows 7 SDK 7.1

Since I already do quite a bit of development on my machine I was lucky to already have VS 2013 & the Windows 7 SDK 7.1 installed.

Next, go ahead and open up the "Windows SDK 7.1 Command Prompt" from the start menu.

For the next steps, Vijay Kumar has provided some excellent steps here:

Brief procedure for some one who might be interested.
My code was in the directory C:\tdameri\xlsxcplus\test1
I used the Windows 7 SDK 7.1 command prompt (you need to have Windows 7.1 SDK installed)
Commands in the command prompt
setenv /x86
cd C:\tdameri\xlsxcplus\test1
npm install edge
cd C:\tdameri\xlsxcplus\test1\node_modules\edge
nw-gyp configure --target=v0.8.4
nw-gyp build

The build happened without errors.

Copy the edge.node file from edge\build\release folder
and replace the edge.node files the edge\lib folder.

Unfortunately it wasn't so simple as it appeared my corporate proxy was blocking or throwing 403 errors when nw-gyp was attempting to pull the NW.js headers from the Amazon AWS site.  After looking through a bit of the code,  I discovered "install.js" did in fact contain proxy support:

// basic support for a proxy server
var proxyUrl = gyp.opts.proxy
            || process.env.http_proxy
            || process.env.HTTP_PROXY
            || process.env.npm_config_proxy
Woohoo!  I was able able to simply SET my "HTTP_PROXY" environment variable and move forward.  A quick note is that Vijay was building for x86 while I was building for x64, therefore, the only change I needed from his steps was modifying the first line to:

setenv /x64

At this point, my custom "edge.node" was building properly:


Following Vijay's steps above, I found that even though we have build "edge.node," we now need to copy it from:

node_modules/edge/build/Release/edge.node

Over to:

node_modules/edge/lib/native/win32/x64/

Speaking of my old friend Vijay--it looked like he didn't realize which folder to copy the newly compiled binary to.  In my case, I figured if I was running NW.js v0.12.3--why not copy it over to the "0.12.0" directory?

Thinking I was in the clear I attempted firing up NW.js and discovered to my horror:


 Boo!  As it turns out, the error does in fact state Edge appears to be looking for a "v1.2.0" folder.  Digging into the Edge.js code more, I discovered the following under "node_modules/edge/lib/edge.js":

var versionMap = [
    [ /^0\.8\./, '0.8.22' ],
    [ /^0\.10\./, '0.10.0' ],
    [ /^0\.12\./, '0.12.0' ],
    [ /^4\./, '4.1.1' ]
];

Aha!  So since Edge is looking to match v1.2.0, I went ahead and modified the above to so:

var versionMap = [
    [ /^0\.8\./, '0.8.22' ],
    [ /^0\.10\./, '0.10.0' ],
    [ /^0\.12\./, '0.12.0' ],
    [ /^4\./, '4.1.1' ],
    [ /^1\.2\./, '1.2.0' ]
];

Then proceeded to create the following directory:

node_modules/edge/lib/native/win32/x64/1.2.0

And finally dumped all the files from the "0.12.0" files into this folder.  After copying the files over, boomzy:


Hope this saves someone a headache.

-CJ